{"id": "internals:internals-tracer-trace-child-tasks", "page": "internals", "ref": "internals-tracer-trace-child-tasks", "title": "Tracing child tasks", "content": "If your code uses a mechanism such as  asyncio.gather()  to execute code in additional tasks you may find that some of the traces are missing from the display. \n                 You can use the  trace_child_tasks()  context manager to ensure these child tasks are correctly handled. \n                 from datasette import tracer\n\nwith tracer.trace_child_tasks():\n    results = await asyncio.gather(\n        # ... async tasks here\n    ) \n                 This example uses the  register_routes()  plugin hook to add a page at  /parallel-queries  which executes two SQL queries in parallel using  asyncio.gather()  and returns their results. \n                 from datasette import hookimpl\nfrom datasette import tracer\n\n\n@hookimpl\ndef register_routes():\n    async def parallel_queries(datasette):\n        db = datasette.get_database()\n        with tracer.trace_child_tasks():\n            one, two = await asyncio.gather(\n                db.execute(\"select 1\"),\n                db.execute(\"select 2\"),\n            )\n        return Response.json(\n            {\n                \"one\": one.single_value(),\n                \"two\": two.single_value(),\n            }\n        )\n\n    return [\n        (r\"/parallel-queries$\", parallel_queries),\n    ] \n                 Adding  ?_trace=1  will show that the trace covers both of those child tasks.", "breadcrumbs": "[\"Internals for plugins\", \"datasette.tracer\"]", "references": "[]"}
{"id": "full_text_search:configuring-fts-using-csvs-to-sqlite", "page": "full_text_search", "ref": "configuring-fts-using-csvs-to-sqlite", "title": "Configuring FTS using csvs-to-sqlite", "content": "If your data starts out in CSV files, you can use Datasette's companion tool  csvs-to-sqlite  to convert that file into a SQLite database and enable full-text search on specific columns. For a file called  items.csv  where you want full-text search to operate against the  name  and  description  columns you would run the following: \n                 $ csvs-to-sqlite items.csv items.db -f name -f description", "breadcrumbs": "[\"Full-text search\", \"Enabling full-text search for a SQLite table\"]", "references": "[{\"href\": \"https://github.com/simonw/csvs-to-sqlite\", \"label\": \"csvs-to-sqlite\"}]"}
{"id": "performance:http-caching", "page": "performance", "ref": "http-caching", "title": "HTTP caching", "content": "If your database is immutable and guaranteed not to change, you can gain major performance improvements from Datasette by enabling HTTP caching. \n             This can work at two different levels. First, it can tell browsers to cache the results of queries and serve future requests from the browser cache. \n             More significantly, it allows you to run Datasette behind a caching proxy such as  Varnish  or use a cache provided by a hosted service such as  Fastly  or  Cloudflare . This can provide incredible speed-ups since a query only needs to be executed by Datasette the first time it is accessed - all subsequent hits can then be served by the cache. \n             Using a caching proxy in this way could enable a Datasette-backed visualization to serve thousands of hits a second while running Datasette itself on extremely inexpensive hosting. \n             Datasette's integration with HTTP caches can be enabled using a combination of configuration options and query string arguments. \n             The  default_cache_ttl  setting sets the default HTTP cache TTL for all Datasette pages. This is 5 seconds unless you change it - you can set it to 0 if you wish to disable HTTP caching entirely. \n             You can also change the cache timeout on a per-request basis using the  ?_ttl=10  query string parameter. This can be useful when you are working with the Datasette JSON API - you may decide that a specific query can be cached for a longer time, or maybe you need to set  ?_ttl=0  for some requests for example if you are running a SQL  order by random()  query.", "breadcrumbs": "[\"Performance and caching\"]", "references": "[{\"href\": \"https://varnish-cache.org/\", \"label\": \"Varnish\"}, {\"href\": \"https://www.fastly.com/\", \"label\": \"Fastly\"}, {\"href\": \"https://www.cloudflare.com/\", \"label\": \"Cloudflare\"}]"}
{"id": "writing_plugins:writing-plugins-static-assets", "page": "writing_plugins", "ref": "writing-plugins-static-assets", "title": "Static assets", "content": "If your plugin has a  static/  directory, Datasette will automatically configure itself to serve those static assets from the following path: \n             /-/static-plugins/NAME_OF_PLUGIN_PACKAGE/yourfile.js \n             Use the  datasette.urls.static_plugins(plugin_name, path)  method to generate URLs to that asset that take the  base_url  setting into account, see  datasette.urls . \n             To bundle the static assets for a plugin in the package that you publish to PyPI, add the following to the plugin's  setup.py : \n             package_data = (\n    {\n        \"datasette_plugin_name\": [\n            \"static/plugin.js\",\n        ],\n    },\n) \n             Where  datasette_plugin_name  is the name of the plugin package (note that it uses underscores, not hyphens) and  static/plugin.js  is the path within that package to the static file. \n             datasette-cluster-map  is a useful example of a plugin that includes packaged static assets in this way.", "breadcrumbs": "[\"Writing plugins\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette-cluster-map\", \"label\": \"datasette-cluster-map\"}]"}
{"id": "writing_plugins:writing-plugins-custom-templates", "page": "writing_plugins", "ref": "writing-plugins-custom-templates", "title": "Custom templates", "content": "If your plugin has a  templates/  directory, Datasette will attempt to load templates from that directory before it uses its own default templates. \n             The priority order for template loading is: \n             \n                 \n                     templates from the  --template-dir  argument, if specified \n                 \n                 \n                     templates from the  templates/  directory in any installed plugins \n                 \n                 \n                     default templates that ship with Datasette \n                 \n             \n             See  Custom pages and templates  for more details on how to write custom templates, including which filenames to use to customize which parts of the Datasette UI. \n             Templates should be bundled for distribution using the same  package_data  mechanism in  setup.py  described for static assets above, for example: \n             package_data = (\n    {\n        \"datasette_plugin_name\": [\n            \"templates/my_template.html\",\n        ],\n    },\n) \n             You can also use wildcards here such as  templates/*.html . See  datasette-edit-schema  for an example of this pattern.", "breadcrumbs": "[\"Writing plugins\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette-edit-schema\", \"label\": \"datasette-edit-schema\"}]"}
{"id": "testing_plugins:testing-plugins-pytest-httpx", "page": "testing_plugins", "ref": "testing-plugins-pytest-httpx", "title": "Testing outbound HTTP calls with pytest-httpx", "content": "If your plugin makes outbound HTTP calls - for example datasette-auth-github or datasette-import-table - you may need to mock those HTTP requests in your tests. \n             The  pytest-httpx  package is a useful library for mocking calls. It can be tricky to use with Datasette though since it mocks all HTTPX requests, and Datasette's own testing mechanism uses HTTPX internally. \n             To avoid breaking your tests, you can return  [\"localhost\"]  from the  non_mocked_hosts()  fixture. \n             As an example, here's a very simple plugin which executes an HTTP response and returns the resulting content: \n             from datasette import hookimpl\nfrom datasette.utils.asgi import Response\nimport httpx\n\n\n@hookimpl\ndef register_routes():\n    return [\n        (r\"^/-/fetch-url$\", fetch_url),\n    ]\n\n\nasync def fetch_url(datasette, request):\n    if request.method == \"GET\":\n        return Response.html(\n            \"\"\"\n            <form action=\"/-/fetch-url\" method=\"post\">\n            <input type=\"hidden\" name=\"csrftoken\" value=\"{}\">\n            <input name=\"url\"><input type=\"submit\">\n        </form>\"\"\".format(\n                request.scope[\"csrftoken\"]()\n            )\n        )\n    vars = await request.post_vars()\n    url = vars[\"url\"]\n    return Response.text(httpx.get(url).text) \n             Here's a test for that plugin that mocks the HTTPX outbound request: \n             from datasette.app import Datasette\nimport pytest\n\n\n@pytest.fixture\ndef non_mocked_hosts():\n    # This ensures httpx-mock will not affect Datasette's own\n    # httpx calls made in the tests by datasette.client:\n    return [\"localhost\"]\n\n\nasync def test_outbound_http_call(httpx_mock):\n    httpx_mock.add_response(\n        url=\"https://www.example.com/\",\n        text=\"Hello world\",\n    )\n    datasette = Datasette([], memory=True)\n    response = await datasette.client.post(\n        \"/-/fetch-url\",\n        data={\"url\": \"https://www.example.com/\"},\n    )\n    assert response.text == \"Hello world\"\n\n    outbound_request = httpx_mock.get_request()\n    assert (\n        outbound_request.url == \"https://www.example.com/\"\n    )", "breadcrumbs": "[\"Testing plugins\"]", "references": "[{\"href\": \"https://pypi.org/project/pytest-httpx/\", \"label\": \"pytest-httpx\"}]"}
{"id": "changelog:id42", "page": "changelog", "ref": "id42", "title": "0.51.1 (2020-10-31)", "content": "Improvements to the new  Binary data  documentation page.", "breadcrumbs": "[\"Changelog\"]", "references": "[]"}
{"id": "internals:internals-response-asgi-send", "page": "internals", "ref": "internals-response-asgi-send", "title": "Returning a response with .asgi_send(send)", "content": "In most cases you will return  Response  objects from your own view functions. You can also use a  Response  instance to respond at a lower level via ASGI, for example if you are writing code that uses the  asgi_wrapper(datasette)  hook. \n                 Create a  Response  object and then use  await response.asgi_send(send) , passing the ASGI  send  function. For example: \n                 async def require_authorization(scope, receive, send):\n    response = Response.text(\n        \"401 Authorization Required\",\n        headers={\n            \"www-authenticate\": 'Basic realm=\"Datasette\", charset=\"UTF-8\"'\n        },\n        status=401,\n    )\n    await response.asgi_send(send)", "breadcrumbs": "[\"Internals for plugins\", \"Response class\"]", "references": "[]"}
{"id": "changelog:id17", "page": "changelog", "ref": "id17", "title": "0.61 (2022-03-23)", "content": "In preparation for Datasette 1.0, this release includes two potentially backwards-incompatible changes. Hashed URL mode has been moved to a separate plugin, and the way Datasette generates URLs to databases and tables with special characters in their name such as  /  and  .  has changed. \n             Datasette also now requires Python 3.7 or higher. \n             \n                 \n                     URLs within Datasette now use a different encoding scheme for tables or databases that include \"special\" characters outside of the range of  a-zA-Z0-9_- . This scheme is explained here:  Tilde encoding . ( #1657 ) \n                 \n                 \n                     Removed hashed URL mode from Datasette. The new  datasette-hashed-urls  plugin can be used to achieve the same result, see  datasette-hashed-urls  for details. ( #1661 ) \n                 \n                 \n                     Databases can now have a custom path within the Datasette instance that is independent of the database name, using the  db.route  property. ( #1668 ) \n                 \n                 \n                     Datasette is now covered by a  Code of Conduct . ( #1654 ) \n                 \n                 \n                     Python 3.6 is no longer supported. ( #1577 ) \n                 \n                 \n                     Tests now run against Python 3.11-dev. ( #1621 ) \n                 \n                 \n                     New  datasette.ensure_permissions(actor, permissions)  internal method for checking multiple permissions at once. ( #1675 ) \n                 \n                 \n                     New  datasette.check_visibility(actor, action, resource=None)  internal method for checking if a user can see a resource that would otherwise be invisible to unauthenticated users. ( #1678 ) \n                 \n                 \n                     Table and row HTML pages now include a  <link rel=\"alternate\" type=\"application/json+datasette\" href=\"...\">  element and return a  Link: URL; rel=\"alternate\"; type=\"application/json+datasette\"  HTTP header pointing to the JSON version of those pages. ( #1533 ) \n                 \n                 \n                     Access-Control-Expose-Headers: Link  is now added to the CORS headers, allowing remote JavaScript to access that header. \n                 \n                 \n                     Canned queries are now shown at the top of the database page, directly below the SQL editor. Previously they were shown at the bottom, below the list of tables. ( #1612 ) \n                 \n                 \n                     Datasette now has a default favicon. ( #1603 ) \n                 \n                 \n                     sqlite_stat  tables are now hidden by default. ( #1587 ) \n                 \n                 \n                     SpatiaLite tables  data_licenses ,  KNN  and  KNN2  are now hidden by default. ( #1601 ) \n                 \n                 \n                     SQL query tracing mechanism now works for queries executed in  asyncio  sub-tasks, such as those created by  asyncio.gather() . ( #1576 ) \n                 \n                 \n                     datasette.tracer  mechanism is now documented. \n                 \n                 \n                     Common Datasette symbols can now be imported directly from the top-level  datasette  package, see  Import shortcuts . Those symbols are  Response ,  Forbidden ,  NotFound ,  hookimpl ,  actor_matches_allow . ( #957 ) \n                 \n                 \n                     /-/versions  page now returns additional details for libraries used by SpatiaLite. ( #1607 ) \n                 \n                 \n                     Documentation now links to the  Datasette Tutorials . \n                 \n                 \n                     Datasette will now also look for SpatiaLite in  /opt/homebrew  - thanks, Dan Peterson. ( #1649 ) \n                 \n                 \n                     Fixed bug where  custom pages  did not work on Windows. Thanks, Robert Christie. ( #1545 ) \n                 \n                 \n                     Fixed error caused when a table had a column named  n . ( #1228 )", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/1657\", \"label\": \"#1657\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1661\", \"label\": \"#1661\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1668\", \"label\": \"#1668\"}, {\"href\": \"https://github.com/simonw/datasette/blob/main/CODE_OF_CONDUCT.md\", \"label\": \"Code of Conduct\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1654\", \"label\": \"#1654\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1577\", \"label\": \"#1577\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1621\", \"label\": \"#1621\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1675\", \"label\": \"#1675\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1678\", \"label\": \"#1678\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1533\", \"label\": \"#1533\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1612\", \"label\": \"#1612\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1603\", \"label\": \"#1603\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1587\", \"label\": \"#1587\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1601\", \"label\": \"#1601\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1576\", \"label\": \"#1576\"}, {\"href\": \"https://github.com/simonw/datasette/issues/957\", \"label\": \"#957\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1607\", \"label\": \"#1607\"}, {\"href\": \"https://datasette.io/tutorials\", \"label\": \"Datasette Tutorials\"}, {\"href\": \"https://github.com/simonw/datasette/pull/1649\", \"label\": \"#1649\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1545\", \"label\": \"#1545\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1228\", \"label\": \"#1228\"}]"}
{"id": "settings:setting-truncate-cells-html", "page": "settings", "ref": "setting-truncate-cells-html", "title": "truncate_cells_html", "content": "In the HTML table view, truncate any strings that are longer than this value.\n                    The full value will still be available in CSV, JSON and on the individual row\n                    HTML page. Set this to 0 to disable truncation. \n                 datasette mydatabase.db --setting truncate_cells_html 0", "breadcrumbs": "[\"Settings\", \"Settings\"]", "references": "[]"}
{"id": "cli-reference:cli-help-install-help", "page": "cli-reference", "ref": "cli-help-install-help", "title": "datasette install", "content": "Install new Datasette plugins. This command works like  pip install  but ensures that your plugins will be installed into the same environment as Datasette. \n             This command: \n             datasette install datasette-cluster-map \n             Would install the  datasette-cluster-map  plugin. \n             [[[cog\nhelp([\"install\", \"--help\"]) \n             ]]] \n             Usage: datasette install [OPTIONS] PACKAGES...\n\n  Install plugins and packages from PyPI into the same environment as Datasette\n\nOptions:\n  -U, --upgrade  Upgrade packages to latest version\n  --help         Show this message and exit. \n             [[[end]]]", "breadcrumbs": "[\"CLI reference\"]", "references": "[{\"href\": \"https://datasette.io/plugins/datasette-cluster-map\", \"label\": \"datasette-cluster-map\"}]"}
{"id": "internals:internals-database", "page": "internals", "ref": "internals-database", "title": "Database class", "content": "Instances of the  Database  class can be used to execute queries against attached SQLite databases, and to run introspection against their schemas.", "breadcrumbs": "[\"Internals for plugins\"]", "references": "[]"}
{"id": "changelog:javascript-modules", "page": "changelog", "ref": "javascript-modules", "title": "JavaScript modules", "content": "JavaScript modules  were introduced in ECMAScript 2015 and provide native browser support for the  import  and  export  keywords. \n                 To use modules, JavaScript needs to be included in  <script>  tags with a  type=\"module\"  attribute. \n                 Datasette now has the ability to output  <script type=\"module\">  in places where you may wish to take advantage of modules. The  extra_js_urls  option described in  Custom CSS and JavaScript  can now be used with modules, and module support is also available for the  extra_body_script()  plugin hook. ( #1186 ,  #1187 ) \n                 datasette-leaflet-freedraw  is the first example of a Datasette plugin that takes advantage of the new support for JavaScript modules. See  Drawing shapes on a map to query a SpatiaLite database  for more on this plugin.", "breadcrumbs": "[\"Changelog\", \"0.54 (2021-01-25)\"]", "references": "[{\"href\": \"https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules\", \"label\": \"JavaScript modules\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1186\", \"label\": \"#1186\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1187\", \"label\": \"#1187\"}, {\"href\": \"https://datasette.io/plugins/datasette-leaflet-freedraw\", \"label\": \"datasette-leaflet-freedraw\"}, {\"href\": \"https://simonwillison.net/2021/Jan/24/drawing-shapes-spatialite/\", \"label\": \"Drawing shapes on a map to query a SpatiaLite database\"}]"}
{"id": "plugin_hooks:plugin-hook-render-cell", "page": "plugin_hooks", "ref": "plugin-hook-render-cell", "title": "render_cell(row, value, column, table, database, datasette)", "content": "Lets you customize the display of values within table cells in the HTML table view. \n             \n                 \n                     row  -  sqlite.Row \n                     \n                         The SQLite row object that the value being rendered is part of \n                     \n                 \n                 \n                     value  - string, integer, float, bytes or None \n                     \n                         The value that was loaded from the database \n                     \n                 \n                 \n                     column  - string \n                     \n                         The name of the column being rendered \n                     \n                 \n                 \n                     table  - string or None \n                     \n                         The name of the table - or  None  if this is a custom SQL query \n                     \n                 \n                 \n                     database  - string \n                     \n                         The name of the database \n                     \n                 \n                 \n                     datasette  -  Datasette class \n                     \n                         You can use this to access plugin configuration options via  datasette.plugin_config(your_plugin_name) , or to execute SQL queries. \n                     \n                 \n             \n             If your hook returns  None , it will be ignored. Use this to indicate that your hook is not able to custom render this particular value. \n             If the hook returns a string, that string will be rendered in the table cell. \n             If you want to return HTML markup you can do so by returning a  jinja2.Markup  object. \n             You can also return an awaitable function which returns a value. \n             Datasette will loop through all available  render_cell  hooks and display the value returned by the first one that does not return  None . \n             Here is an example of a custom  render_cell()  plugin which looks for values that are a JSON string matching the following format: \n             {\"href\": \"https://www.example.com/\", \"label\": \"Name\"} \n             If the value matches that pattern, the plugin returns an HTML link element: \n             from datasette import hookimpl\nimport markupsafe\nimport json\n\n\n@hookimpl\ndef render_cell(value):\n    # Render {\"href\": \"...\", \"label\": \"...\"} as link\n    if not isinstance(value, str):\n        return None\n    stripped = value.strip()\n    if not (\n        stripped.startswith(\"{\") and stripped.endswith(\"}\")\n    ):\n        return None\n    try:\n        data = json.loads(value)\n    except ValueError:\n        return None\n    if not isinstance(data, dict):\n        return None\n    if set(data.keys()) != {\"href\", \"label\"}:\n        return None\n    href = data[\"href\"]\n    if not (\n        href.startswith(\"/\")\n        or href.startswith(\"http://\")\n        or href.startswith(\"https://\")\n    ):\n        return None\n    return markupsafe.Markup(\n        '<a href=\"{href}\">{label}</a>'.format(\n            href=markupsafe.escape(data[\"href\"]),\n            label=markupsafe.escape(data[\"label\"] or \"\")\n            or \"&nbsp;\",\n        )\n    ) \n             Examples:  datasette-render-binary ,  datasette-render-markdown ,  datasette-json-html", "breadcrumbs": "[\"Plugin hooks\"]", "references": "[{\"href\": \"https://datasette.io/plugins/datasette-render-binary\", \"label\": \"datasette-render-binary\"}, {\"href\": \"https://datasette.io/plugins/datasette-render-markdown\", \"label\": \"datasette-render-markdown\"}, {\"href\": \"https://datasette.io/plugins/datasette-json-html\", \"label\": \"datasette-json-html\"}]"}
{"id": "internals:database-execute-write-script", "page": "internals", "ref": "database-execute-write-script", "title": "await db.execute_write_script(sql, block=True)", "content": "Like  execute_write()  but can be used to send multiple SQL statements in a single string separated by semicolons, using the  sqlite3   conn.executescript()  method.", "breadcrumbs": "[\"Internals for plugins\", \"Database class\"]", "references": "[{\"href\": \"https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.executescript\", \"label\": \"conn.executescript()\"}]"}
{"id": "internals:database-execute-write-many", "page": "internals", "ref": "database-execute-write-many", "title": "await db.execute_write_many(sql, params_seq, block=True)", "content": "Like  execute_write()  but uses the  sqlite3   conn.executemany()  method. This will efficiently execute the same SQL statement against each of the parameters in the  params_seq  iterator, for example: \n                 await db.execute_write_many(\n    \"insert into characters (id, name) values (?, ?)\",\n    [(1, \"Melanie\"), (2, \"Selma\"), (2, \"Viktor\")],\n)", "breadcrumbs": "[\"Internals for plugins\", \"Database class\"]", "references": "[{\"href\": \"https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.executemany\", \"label\": \"conn.executemany()\"}]"}
{"id": "changelog:other-small-fixes", "page": "changelog", "ref": "other-small-fixes", "title": "Other small fixes", "content": "Made several performance improvements to the database schema introspection code that runs when Datasette first starts up. ( #1555 ) \n                     \n                     \n                         Label columns detected for foreign keys are now case-insensitive, so  Name  or  TITLE  will be detected in the same way as  name  or  title . ( #1544 ) \n                     \n                     \n                         Upgraded Pluggy dependency to 1.0. ( #1575 ) \n                     \n                     \n                         Now using  Plausible analytics  for the Datasette documentation. \n                     \n                     \n                         explain query plan  is now allowed with varying amounts of whitespace in the query. ( #1588 ) \n                     \n                     \n                         New  CLI reference  page showing the output of  --help  for each of the  datasette  sub-commands. This lead to several small improvements to the help copy. ( #1594 ) \n                     \n                     \n                         Fixed bug where writable canned queries could not be used with custom templates.  ( #1547 ) \n                     \n                     \n                         Improved fix for a bug where columns with a underscore prefix could result in unnecessary hidden form fields. ( #1527 )", "breadcrumbs": "[\"Changelog\", \"0.60 (2022-01-13)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/1555\", \"label\": \"#1555\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1544\", \"label\": \"#1544\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1575\", \"label\": \"#1575\"}, {\"href\": \"https://plausible.io/\", \"label\": \"Plausible analytics\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1588\", \"label\": \"#1588\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1594\", \"label\": \"#1594\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1547\", \"label\": \"#1547\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1527\", \"label\": \"#1527\"}]"}
{"id": "internals:internals", "page": "internals", "ref": "internals", "title": "Internals for plugins", "content": "Many  Plugin hooks  are passed objects that provide access to internal Datasette functionality. The interface to these objects should not be considered stable with the exception of methods that are documented here.", "breadcrumbs": "[]", "references": "[]"}
{"id": "settings:setting-num-sql-threads", "page": "settings", "ref": "setting-num-sql-threads", "title": "num_sql_threads", "content": "Maximum number of threads in the thread pool Datasette uses to execute SQLite queries. Defaults to 3. \n                 datasette mydatabase.db --setting num_sql_threads 10 \n                 Setting this to 0 turns off threaded SQL queries entirely - useful for environments that do not support threading such as  Pyodide .", "breadcrumbs": "[\"Settings\", \"Settings\"]", "references": "[{\"href\": \"https://pyodide.org/\", \"label\": \"Pyodide\"}]"}
{"id": "metadata:per-database-and-per-table-metadata", "page": "metadata", "ref": "per-database-and-per-table-metadata", "title": "Per-database and per-table metadata", "content": "Metadata at the top level of the JSON will be shown on the index page and in the\n                footer on every page of the site. The license and source is expected to apply to\n                all of your data. \n             You can also provide metadata at the per-database or per-table level, like this: \n             {\n    \"databases\": {\n        \"database1\": {\n            \"source\": \"Alternative source\",\n            \"source_url\": \"http://example.com/\",\n            \"tables\": {\n                \"example_table\": {\n                    \"description_html\": \"Custom <em>table</em> description\",\n                    \"license\": \"CC BY 3.0 US\",\n                    \"license_url\": \"https://creativecommons.org/licenses/by/3.0/us/\"\n                }\n            }\n        }\n    }\n} \n             Each of the top-level metadata fields can be used at the database and table level.", "breadcrumbs": "[\"Metadata\"]", "references": "[]"}
{"id": "changelog:id100", "page": "changelog", "ref": "id100", "title": "0.23.2 (2018-07-07)", "content": "Minor bugfix and documentation release. \n             \n                 \n                     CSV export now respects  --cors , fixes  #326 \n                 \n                 \n                     Installation instructions , including docker image - closes  #328 \n                 \n                 \n                     Fix for row pages for tables with / in, closes  #325", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/326\", \"label\": \"#326\"}, {\"href\": \"https://github.com/simonw/datasette/issues/328\", \"label\": \"#328\"}, {\"href\": \"https://github.com/simonw/datasette/issues/325\", \"label\": \"#325\"}]"}
{"id": "changelog:id104", "page": "changelog", "ref": "id104", "title": "0.23.1 (2018-06-21)", "content": "Minor bugfix release. \n             \n                 \n                     Correctly display empty strings in HTML table, closes  #314 \n                 \n                 \n                     Allow \".\" in database filenames, closes  #302 \n                 \n                 \n                     404s ending in slash redirect to remove that slash, closes  #309 \n                 \n                 \n                     Fixed incorrect display of compound primary keys with foreign key\n                        references. Closes  #319 \n                 \n                 \n                     Docs + example of canned SQL query using || concatenation. Closes  #321 \n                 \n                 \n                     Correctly display facets with value of 0 - closes  #318 \n                 \n                 \n                     Default 'expand labels' to checked in CSV advanced export", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/314\", \"label\": \"#314\"}, {\"href\": \"https://github.com/simonw/datasette/issues/302\", \"label\": \"#302\"}, {\"href\": \"https://github.com/simonw/datasette/issues/309\", \"label\": \"#309\"}, {\"href\": \"https://github.com/simonw/datasette/issues/319\", \"label\": \"#319\"}, {\"href\": \"https://github.com/simonw/datasette/issues/321\", \"label\": \"#321\"}, {\"href\": \"https://github.com/simonw/datasette/issues/318\", \"label\": \"#318\"}]"}
{"id": "json_api:json-api-discover-alternate", "page": "json_api", "ref": "json-api-discover-alternate", "title": "Discovering the JSON for a page", "content": "Most of the HTML pages served by Datasette provide a mechanism for discovering their JSON equivalents using the HTML  link  mechanism. \n             You can find this near the top of the source code of those pages, looking like this: \n             <link rel=\"alternate\"\n  type=\"application/json+datasette\"\n  href=\"https://latest.datasette.io/fixtures/sortable.json\"> \n             The JSON URL is also made available in a  Link  HTTP header for the page: \n             Link: https://latest.datasette.io/fixtures/sortable.json; rel=\"alternate\"; type=\"application/json+datasette\"", "breadcrumbs": "[\"JSON API\"]", "references": "[]"}
{"id": "changelog:id133", "page": "changelog", "ref": "id133", "title": "0.20 (2018-04-20)", "content": "Mostly new work on the  Plugins  mechanism: plugins can now bundle static assets and custom templates, and  datasette publish  has a new  --install=name-of-plugin  option. \n             \n                 \n                     Add col-X classes to HTML table on custom query page \n                 \n                 \n                     Fixed out-dated template in documentation \n                 \n                 \n                     Plugins can now bundle custom templates,  #224 \n                 \n                 \n                     Added /-/metadata /-/plugins /-/inspect,  #225 \n                 \n                 \n                     Documentation for --install option, refs  #223 \n                 \n                 \n                     Datasette publish/package --install option,  #223 \n                 \n                 \n                     Fix for plugins in Python 3.5,  #222 \n                 \n                 \n                     New plugin hooks: extra_css_urls() and extra_js_urls(),  #214 \n                 \n                 \n                     /-/static-plugins/PLUGIN_NAME/ now serves static/ from plugins \n                 \n                 \n                     <th> now gets class=\"col-X\" - plus added col-X documentation \n                 \n                 \n                     Use to_css_class for table cell column classes \n                     This ensures that columns with spaces in the name will still\n                        generate usable CSS class names. Refs  #209 \n                 \n                 \n                     Add column name classes to <td>s, make PK bold [Russ Garrett] \n                 \n                 \n                     Don't duplicate simple primary keys in the link column [Russ Garrett] \n                     When there's a simple (single-column) primary key, it looks weird to\n                        duplicate it in the link column. \n                     This change removes the second PK column and treats the link column as\n                        if it were the PK column from a header/sorting perspective. \n                 \n                 \n                     Correct escaping for HTML display of row links [Russ Garrett] \n                 \n                 \n                     Longer time limit for test_paginate_compound_keys \n                     It was failing intermittently in Travis - see  #209 \n                 \n                 \n                     Use application/octet-stream for downloadable databases \n                 \n                 \n                     Updated PyPI classifiers \n                 \n                 \n                     Updated PyPI link to pypi.org", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/224\", \"label\": \"#224\"}, {\"href\": \"https://github.com/simonw/datasette/issues/225\", \"label\": \"#225\"}, {\"href\": \"https://github.com/simonw/datasette/issues/223\", \"label\": \"#223\"}, {\"href\": \"https://github.com/simonw/datasette/issues/223\", \"label\": \"#223\"}, {\"href\": \"https://github.com/simonw/datasette/issues/222\", \"label\": \"#222\"}, {\"href\": \"https://github.com/simonw/datasette/issues/214\", \"label\": \"#214\"}, {\"href\": \"https://github.com/simonw/datasette/issues/209\", \"label\": \"#209\"}, {\"href\": \"https://github.com/simonw/datasette/issues/209\", \"label\": \"#209\"}]"}
{"id": "sql_queries:canned-queries-magic-parameters", "page": "sql_queries", "ref": "canned-queries-magic-parameters", "title": "Magic parameters", "content": "Named parameters that start with an underscore are special: they can be used to automatically add values created by Datasette that are not contained in the incoming form fields or query string. \n                 These magic parameters are only supported for canned queries: to avoid security issues (such as queries that extract the user's private cookies) they are not available to SQL that is executed by the user as a custom SQL query. \n                 Available magic parameters are: \n                 \n                     \n                         _actor_*  - e.g.  _actor_id ,  _actor_name \n                         \n                             Fields from the currently authenticated  Actors . \n                         \n                     \n                     \n                         _header_*  - e.g.  _header_user_agent \n                         \n                             Header from the incoming HTTP request. The key should be in lower case and with hyphens converted to underscores e.g.  _header_user_agent  or  _header_accept_language . \n                         \n                     \n                     \n                         _cookie_*  - e.g.  _cookie_lang \n                         \n                             The value of the incoming cookie of that name. \n                         \n                     \n                     \n                         _now_epoch \n                         \n                             The number of seconds since the Unix epoch. \n                         \n                     \n                     \n                         _now_date_utc \n                         \n                             The date in UTC, e.g.  2020-06-01 \n                         \n                     \n                     \n                         _now_datetime_utc \n                         \n                             The ISO 8601 datetime in UTC, e.g.  2020-06-24T18:01:07Z \n                         \n                     \n                     \n                         _random_chars_*  - e.g.  _random_chars_128 \n                         \n                             A random string of characters of the specified length. \n                         \n                     \n                 \n                 Here's an example configuration (this time using  metadata.yaml  since that provides better support for multi-line SQL queries) that adds a message from the authenticated user, storing various pieces of additional metadata using magic parameters: \n                 databases:\n  mydatabase:\n    queries:\n      add_message:\n        allow:\n          id: \"*\"\n        sql: |-\n          INSERT INTO messages (\n            user_id, message, datetime\n          ) VALUES (\n            :_actor_id, :message, :_now_datetime_utc\n          )\n        write: true \n                 The form presented at  /mydatabase/add_message  will have just a field for  message  - the other parameters will be populated by the magic parameter mechanism. \n                 Additional custom magic parameters can be added by plugins using the  register_magic_parameters(datasette)  hook.", "breadcrumbs": "[\"Running SQL queries\", \"Canned queries\"]", "references": "[]"}
{"id": "changelog:id63", "page": "changelog", "ref": "id63", "title": "0.39 (2020-03-24)", "content": "New  base_url  configuration setting for serving up the correct links while running Datasette under a different URL prefix. ( #394 ) \n                 \n                 \n                     New metadata settings  \"sort\"  and  \"sort_desc\"  for setting the default sort order for a table. See  Setting a default sort order . ( #702 ) \n                 \n                 \n                     Sort direction arrow now displays by default on the primary key. This means you only have to click once (not twice) to sort in reverse order. ( #677 ) \n                 \n                 \n                     New  await Request(scope, receive).post_vars()  method for accessing POST form variables. ( #700 ) \n                 \n                 \n                     Plugin hooks  documentation now links to example uses of each plugin. ( #709 )", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/394\", \"label\": \"#394\"}, {\"href\": \"https://github.com/simonw/datasette/issues/702\", \"label\": \"#702\"}, {\"href\": \"https://github.com/simonw/datasette/issues/677\", \"label\": \"#677\"}, {\"href\": \"https://github.com/simonw/datasette/issues/700\", \"label\": \"#700\"}, {\"href\": \"https://github.com/simonw/datasette/issues/709\", \"label\": \"#709\"}]"}
{"id": "changelog:id27", "page": "changelog", "ref": "id27", "title": "0.58 (2021-07-14)", "content": "New  datasette --uds /tmp/datasette.sock  option for binding Datasette to a Unix domain socket, see  proxy documentation  ( #1388 ) \n                 \n                 \n                     \"searchmode\": \"raw\"  table metadata option for defaulting a table to executing SQLite full-text search syntax without first escaping it, see  Advanced SQLite search queries . ( #1389 ) \n                 \n                 \n                     New plugin hook:  get_metadata(datasette, key, database, table) , for returning custom metadata for an instance, database or table. Thanks, Brandon Roberts! ( #1384 ) \n                 \n                 \n                     New plugin hook:  skip_csrf(datasette, scope) , for opting out of CSRF protection based on the incoming request. ( #1377 ) \n                 \n                 \n                     The  menu_links() ,  table_actions()  and  database_actions()  plugin hooks all gained a new optional  request  argument providing access to the current request. ( #1371 ) \n                 \n                 \n                     Major performance improvement for Datasette faceting. ( #1394 ) \n                 \n                 \n                     Improved documentation for  Running Datasette behind a proxy  to recommend using  ProxyPreservehost On  with Apache. ( #1387 ) \n                 \n                 \n                     POST  requests to endpoints that do not support that HTTP verb now return a 405 error. \n                 \n                 \n                     db.path  can now be provided as a  pathlib.Path  object, useful when writing unit tests for plugins. Thanks, Chris Amico. ( #1365 )", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/1388\", \"label\": \"#1388\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1389\", \"label\": \"#1389\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1384\", \"label\": \"#1384\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1377\", \"label\": \"#1377\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1371\", \"label\": \"#1371\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1394\", \"label\": \"#1394\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1387\", \"label\": \"#1387\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1365\", \"label\": \"#1365\"}]"}
{"id": "changelog:id120", "page": "changelog", "ref": "id120", "title": "0.21 (2018-05-05)", "content": "New JSON  _shape=  options, the ability to set table  _size=  and a mechanism for searching within specific columns. \n             \n                 \n                     Default tests to using a longer timelimit \n                     Every now and then a test will fail in Travis CI on Python 3.5 because it hit\n                        the default 20ms SQL time limit. \n                     Test fixtures now default to a 200ms time limit, and we only use the 20ms time\n                        limit for the specific test that tests query interruption. This should make\n                        our tests on Python 3.5 in Travis much more stable. \n                 \n                 \n                     Support  _search_COLUMN=text  searches, closes  #237 \n                 \n                 \n                     Show version on  /-/plugins  page, closes  #248 \n                 \n                 \n                     ?_size=max  option, closes  #249 \n                 \n                 \n                     Added  /-/versions  and  /-/versions.json , closes  #244 \n                     Sample output: \n                     {\n  \"python\": {\n    \"version\": \"3.6.3\",\n    \"full\": \"3.6.3 (default, Oct  4 2017, 06:09:38) \\n[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)]\"\n  },\n  \"datasette\": {\n    \"version\": \"0.20\"\n  },\n  \"sqlite\": {\n    \"version\": \"3.23.1\",\n    \"extensions\": {\n      \"json1\": null,\n      \"spatialite\": \"4.3.0a\"\n    }\n  }\n} \n                 \n                 \n                     Renamed  ?_sql_time_limit_ms=  to  ?_timelimit , closes  #242 \n                 \n                 \n                     New  ?_shape=array  option + tweaks to  _shape , closes  #245 \n                     \n                         \n                             Default is now  ?_shape=arrays  (renamed from  lists ) \n                         \n                         \n                             New  ?_shape=array  returns an array of objects as the root object \n                         \n                         \n                             Changed  ?_shape=object  to return the object as the root \n                         \n                         \n                             Updated docs \n                         \n                     \n                 \n                 \n                     FTS tables now detected by  inspect() , closes  #240 \n                 \n                 \n                     New  ?_size=XXX  query string parameter for table view, closes  #229 \n                     Also added documentation for all of the  _special  arguments. \n                     Plus deleted some duplicate logic implementing  _group_count . \n                 \n                 \n                     If  max_returned_rows==page_size , increment  max_returned_rows  - fixes  #230 \n                 \n                 \n                     New  hidden: True  option for table metadata, closes  #239 \n                 \n                 \n                     Hide  idx_*  tables if spatialite detected, closes  #228 \n                 \n                 \n                     Added  class=rows-and-columns  to custom query results table \n                 \n                 \n                     Added CSS class  rows-and-columns  to main table \n                 \n                 \n                     label_column  option in  metadata.json  - closes  #234", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/237\", \"label\": \"#237\"}, {\"href\": \"https://github.com/simonw/datasette/issues/248\", \"label\": \"#248\"}, {\"href\": \"https://github.com/simonw/datasette/issues/249\", \"label\": \"#249\"}, {\"href\": \"https://github.com/simonw/datasette/issues/244\", \"label\": \"#244\"}, {\"href\": \"https://github.com/simonw/datasette/issues/242\", \"label\": \"#242\"}, {\"href\": \"https://github.com/simonw/datasette/issues/245\", \"label\": \"#245\"}, {\"href\": \"https://github.com/simonw/datasette/issues/240\", \"label\": \"#240\"}, {\"href\": \"https://github.com/simonw/datasette/issues/229\", \"label\": \"#229\"}, {\"href\": \"https://github.com/simonw/datasette/issues/230\", \"label\": \"#230\"}, {\"href\": \"https://github.com/simonw/datasette/issues/239\", \"label\": \"#239\"}, {\"href\": \"https://github.com/simonw/datasette/issues/228\", \"label\": \"#228\"}, {\"href\": \"https://github.com/simonw/datasette/issues/234\", \"label\": \"#234\"}]"}
{"id": "changelog:id85", "page": "changelog", "ref": "id85", "title": "0.27 (2019-01-31)", "content": "New command:  datasette plugins  ( documentation ) shows you the currently installed list of plugins. \n                 \n                 \n                     Datasette can now output  newline-delimited JSON  using the new  ?_shape=array&_nl=on  query string option. \n                 \n                 \n                     Added documentation on  The Datasette Ecosystem . \n                 \n                 \n                     Now using Python 3.7.2 as the base for the official  Datasette Docker image .", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"http://ndjson.org/\", \"label\": \"newline-delimited JSON\"}, {\"href\": \"https://hub.docker.com/r/datasetteproject/datasette/\", \"label\": \"Datasette Docker image\"}]"}
{"id": "changelog:id58", "page": "changelog", "ref": "id58", "title": "Smaller changes", "content": "New internals documentation for  Request object  and  Response class . ( #706 ) \n                     \n                     \n                         request.url  now respects the  force_https_urls  config setting. closes ( #781 ) \n                     \n                     \n                         request.args.getlist()  returns  []  if missing. Removed  request.raw_args  entirely. ( #774 ) \n                     \n                     \n                         New  datasette.get_database()  method. \n                     \n                     \n                         Added  _  prefix to many private, undocumented methods of the Datasette class. ( #576 ) \n                     \n                     \n                         Removed the  db.get_outbound_foreign_keys()  method which duplicated the behaviour of  db.foreign_keys_for_table() . \n                     \n                     \n                         New  await datasette.permission_allowed()  method. \n                     \n                     \n                         /-/actor  debugging endpoint for viewing the currently authenticated actor. \n                     \n                     \n                         New  request.cookies  property. \n                     \n                     \n                         /-/plugins  endpoint now shows a list of hooks implemented by each plugin, e.g.  https://latest.datasette.io/-/plugins?all=1 \n                     \n                     \n                         request.post_vars()  method no longer discards empty values. \n                     \n                     \n                         New \"params\" canned query key for explicitly setting named parameters, see  Canned query parameters . ( #797 ) \n                     \n                     \n                         request.args  is now a  MultiParams  object. \n                     \n                     \n                         Fixed a bug with the  datasette plugins  command. ( #802 ) \n                     \n                     \n                         Nicer pattern for using  make_app_client()  in tests. ( #395 ) \n                     \n                     \n                         New  request.actor  property. \n                     \n                     \n                         Fixed broken CSS on nested 404 pages. ( #777 ) \n                     \n                     \n                         New  request.url_vars  property. ( #822 ) \n                     \n                     \n                         Fixed a bug with the  python tests/fixtures.py  command for outputting Datasette's testing fixtures database and plugins. ( #804 ) \n                     \n                     \n                         datasette publish heroku  now deploys using Python 3.8.3. \n                     \n                     \n                         Added a warning that the  register_facet_classes()  hook is unstable and may change in the future. ( #830 ) \n                     \n                     \n                         The  {\"$env\": \"ENVIRONMENT_VARIBALE\"}  mechanism (see  Secret configuration values ) now works with variables inside nested lists. ( #837 )", "breadcrumbs": "[\"Changelog\", \"0.44 (2020-06-11)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/706\", \"label\": \"#706\"}, {\"href\": \"https://github.com/simonw/datasette/issues/781\", \"label\": \"#781\"}, {\"href\": \"https://github.com/simonw/datasette/issues/774\", \"label\": \"#774\"}, {\"href\": \"https://github.com/simonw/datasette/issues/576\", \"label\": \"#576\"}, {\"href\": \"https://latest.datasette.io/-/plugins?all=1\", \"label\": \"https://latest.datasette.io/-/plugins?all=1\"}, {\"href\": \"https://github.com/simonw/datasette/issues/797\", \"label\": \"#797\"}, {\"href\": \"https://github.com/simonw/datasette/issues/802\", \"label\": \"#802\"}, {\"href\": \"https://github.com/simonw/datasette/issues/395\", \"label\": \"#395\"}, {\"href\": \"https://github.com/simonw/datasette/issues/777\", \"label\": \"#777\"}, {\"href\": \"https://github.com/simonw/datasette/issues/822\", \"label\": \"#822\"}, {\"href\": \"https://github.com/simonw/datasette/issues/804\", \"label\": \"#804\"}, {\"href\": \"https://github.com/simonw/datasette/issues/830\", \"label\": \"#830\"}, {\"href\": \"https://github.com/simonw/datasette/issues/837\", \"label\": \"#837\"}]"}
{"id": "changelog:plugins-and-internals", "page": "changelog", "ref": "plugins-and-internals", "title": "Plugins and internals", "content": "New plugin hook:  filters_from_request(request, database, table, datasette) , which runs on the table page and can be used to support new custom query string parameters that modify the SQL query. ( #473 ) \n                     \n                     \n                         Added two additional methods for writing to the database:  await db.execute_write_script(sql, block=True)  and  await db.execute_write_many(sql, params_seq, block=True) . ( #1570 ) \n                     \n                     \n                         The  db.execute_write()  internal method now defaults to blocking until the write operation has completed. Previously it defaulted to queuing the write and then continuing to run code while the write was in the queue. ( #1579 ) \n                     \n                     \n                         Database write connections now execute the  prepare_connection(conn, database, datasette)  plugin hook. ( #1564 ) \n                     \n                     \n                         The  Datasette()  constructor no longer requires the  files=  argument, and is now documented at  Datasette class . ( #1563 ) \n                     \n                     \n                         The tracing feature now traces write queries, not just read queries. ( #1568 ) \n                     \n                     \n                         The query string variables exposed by  request.args  will now include blank strings for arguments such as  foo  in  ?foo=&bar=1  rather than ignoring those parameters entirely. ( #1551 )", "breadcrumbs": "[\"Changelog\", \"0.60 (2022-01-13)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/473\", \"label\": \"#473\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1570\", \"label\": \"#1570\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1579\", \"label\": \"#1579\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1564\", \"label\": \"#1564\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1563\", \"label\": \"#1563\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1568\", \"label\": \"#1568\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1551\", \"label\": \"#1551\"}]"}
{"id": "changelog:plugin-hooks", "page": "changelog", "ref": "plugin-hooks", "title": "Plugin hooks", "content": "New plugin hook:  handle_exception() , for custom handling of exceptions caught by Datasette. ( #1770 ) \n                     \n                     \n                         The  render_cell()  plugin hook is now also passed a  row  argument, representing the  sqlite3.Row  object that is being rendered. ( #1300 ) \n                     \n                     \n                         The  configuration directory  is now stored in  datasette.config_dir , making it available to plugins. Thanks, Chris Amico. ( #1766 )", "breadcrumbs": "[\"Changelog\", \"0.62 (2022-08-14)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/1770\", \"label\": \"#1770\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1300\", \"label\": \"#1300\"}, {\"href\": \"https://github.com/simonw/datasette/pull/1766\", \"label\": \"#1766\"}]"}
{"id": "changelog:id90", "page": "changelog", "ref": "id90", "title": "0.25 (2018-09-19)", "content": "New plugin hooks, improved database view support and an easier way to use more recent versions of SQLite. \n             \n                 \n                     New  publish_subcommand  plugin hook. A plugin can now add additional  datasette publish  publishers in addition to the default  now  and  heroku , both of which have been refactored into default plugins.  publish_subcommand documentation . Closes  #349 \n                 \n                 \n                     New  render_cell  plugin hook. Plugins can now customize how values are displayed in the HTML tables produced by Datasette's browsable interface.  datasette-json-html  and  datasette-render-images  are two new plugins that use this hook.  render_cell documentation . Closes  #352 \n                 \n                 \n                     New  extra_body_script  plugin hook, enabling plugins to provide additional JavaScript that should be added to the page footer.  extra_body_script documentation . \n                 \n                 \n                     extra_css_urls  and  extra_js_urls  hooks now take additional optional parameters, allowing them to be more selective about which pages they apply to.  Documentation . \n                 \n                 \n                     You can now use the  sortable_columns metadata setting  to explicitly enable sort-by-column in the interface for database views, as well as for specific tables. \n                 \n                 \n                     The new  fts_table  and  fts_pk  metadata settings can now be used to  explicitly configure full-text search for a table or a view , even if that table is not directly coupled to the SQLite FTS feature in the database schema itself. \n                 \n                 \n                     Datasette will now use  pysqlite3  in place of the standard library  sqlite3  module if it has been installed in the current environment. This makes it much easier to run Datasette against a more recent version of SQLite, including the just-released  SQLite 3.25.0  which adds window function support. More details on how to use this in  #360 \n                 \n                 \n                     New mechanism that allows  plugin configuration options  to be set using  metadata.json .", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/349\", \"label\": \"#349\"}, {\"href\": \"https://github.com/simonw/datasette-json-html\", \"label\": \"datasette-json-html\"}, {\"href\": \"https://github.com/simonw/datasette-render-images\", \"label\": \"datasette-render-images\"}, {\"href\": \"https://github.com/simonw/datasette/issues/352\", \"label\": \"#352\"}, {\"href\": \"https://github.com/coleifer/pysqlite3\", \"label\": \"pysqlite3\"}, {\"href\": \"https://www.sqlite.org/releaselog/3_25_0.html\", \"label\": \"SQLite 3.25.0\"}, {\"href\": \"https://github.com/simonw/datasette/issues/360\", \"label\": \"#360\"}]"}
{"id": "changelog:documentation", "page": "changelog", "ref": "documentation", "title": "Documentation", "content": "New tutorial:  Cleaning data with sqlite-utils and Datasette . \n                     \n                     \n                         Screenshots in the documentation are now maintained using  shot-scraper , as described in  Automating screenshots for the Datasette documentation using shot-scraper . ( #1844 ) \n                     \n                     \n                         More detailed command descriptions on the  CLI reference  page. ( #1787 ) \n                     \n                     \n                         New documentation on  Running Datasette using OpenRC  - thanks, Adam Simpson. ( #1825 )", "breadcrumbs": "[\"Changelog\", \"0.63 (2022-10-27)\"]", "references": "[{\"href\": \"https://datasette.io/tutorials/clean-data\", \"label\": \"Cleaning data with sqlite-utils and Datasette\"}, {\"href\": \"https://shot-scraper.datasette.io/\", \"label\": \"shot-scraper\"}, {\"href\": \"https://simonwillison.net/2022/Oct/14/automating-screenshots/\", \"label\": \"Automating screenshots for the Datasette documentation using shot-scraper\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1844\", \"label\": \"#1844\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1787\", \"label\": \"#1787\"}, {\"href\": \"https://github.com/simonw/datasette/pull/1825\", \"label\": \"#1825\"}]"}
{"id": "settings:config-dir", "page": "settings", "ref": "config-dir", "title": "Configuration directory mode", "content": "Normally you configure Datasette using command-line options. For a Datasette instance with custom templates, custom plugins, a static directory and several databases this can get quite verbose: \n             $ datasette one.db two.db \\\n    --metadata=metadata.json \\\n    --template-dir=templates/ \\\n    --plugins-dir=plugins \\\n    --static css:css \n             As an alternative to this, you can run Datasette in  configuration directory  mode. Create a directory with the following structure: \n             # In a directory called my-app:\nmy-app/one.db\nmy-app/two.db\nmy-app/metadata.json\nmy-app/templates/index.html\nmy-app/plugins/my_plugin.py\nmy-app/static/my.css \n             Now start Datasette by providing the path to that directory: \n             $ datasette my-app/ \n             Datasette will detect the files in that directory and automatically configure itself using them. It will serve all  *.db  files that it finds, will load  metadata.json  if it exists, and will load the  templates ,  plugins  and  static  folders if they are present. \n             The files that can be included in this directory are as follows. All are optional. \n             \n                 \n                     *.db  (or  *.sqlite3  or  *.sqlite ) - SQLite database files that will be served by Datasette \n                 \n                 \n                     metadata.json  -  Metadata  for those databases -  metadata.yaml  or  metadata.yml  can be used as well \n                 \n                 \n                     inspect-data.json  - the result of running  datasette inspect *.db --inspect-file=inspect-data.json  from the configuration directory - any database files listed here will be treated as immutable, so they should not be changed while Datasette is running \n                 \n                 \n                     settings.json  - settings that would normally be passed using  --setting  - here they should be stored as a JSON object of key/value pairs \n                 \n                 \n                     templates/  - a directory containing  Custom templates \n                 \n                 \n                     plugins/  - a directory containing plugins, see  Writing one-off plugins \n                 \n                 \n                     static/  - a directory containing static files - these will be served from  /static/filename.txt , see  Serving static files", "breadcrumbs": "[\"Settings\"]", "references": "[]"}
{"id": "changelog:features", "page": "changelog", "ref": "features", "title": "Features", "content": "Now tested against Python 3.11. Docker containers used by  datasette publish  and  datasette package  both now use that version of Python. ( #1853 ) \n                     \n                     \n                         --load-extension  option now supports entrypoints. Thanks, Alex Garcia. ( #1789 ) \n                     \n                     \n                         Facet size can now be set per-table with the new  facet_size  table metadata option. ( #1804 ) \n                     \n                     \n                         The  truncate_cells_html  setting now also affects long URLs in columns. ( #1805 ) \n                     \n                     \n                         The non-JavaScript SQL editor textarea now increases height to fit the SQL query. ( #1786 ) \n                     \n                     \n                         Facets are now displayed with better line-breaks in long values. Thanks, Daniel Rech. ( #1794 ) \n                     \n                     \n                         The  settings.json  file used in  Configuration directory mode  is now validated on startup. ( #1816 ) \n                     \n                     \n                         SQL queries can now include leading SQL comments, using  /* ... */  or  -- ...  syntax. Thanks,  Charles Nepote. ( #1860 ) \n                     \n                     \n                         SQL query is now re-displayed when terminated with a time limit error. ( #1819 ) \n                     \n                     \n                         The  inspect data  mechanism is now used to speed up server startup - thanks, Forest Gregg. ( #1834 ) \n                     \n                     \n                         In  Configuration directory mode  databases with filenames ending in  .sqlite  or  .sqlite3  are now automatically added to the Datasette instance. ( #1646 ) \n                     \n                     \n                         Breadcrumb navigation display now respects the current user's permissions. ( #1831 )", "breadcrumbs": "[\"Changelog\", \"0.63 (2022-10-27)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/1853\", \"label\": \"#1853\"}, {\"href\": \"https://github.com/simonw/datasette/pull/1789\", \"label\": \"#1789\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1804\", \"label\": \"#1804\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1805\", \"label\": \"#1805\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1786\", \"label\": \"#1786\"}, {\"href\": \"https://github.com/simonw/datasette/pull/1794\", \"label\": \"#1794\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1816\", \"label\": \"#1816\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1860\", \"label\": \"#1860\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1819\", \"label\": \"#1819\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1834\", \"label\": \"#1834\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1646\", \"label\": \"#1646\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1831\", \"label\": \"#1831\"}]"}
{"id": "publish:cli-publish", "page": "publish", "ref": "cli-publish", "title": "datasette publish", "content": "Once you have created a SQLite database (e.g. using  csvs-to-sqlite ) you can deploy it to a hosting account using a single command. \n             You will need a hosting account with  Heroku  or  Google Cloud . Once you have created your account you will need to install and configure the  heroku  or  gcloud  command-line tools.", "breadcrumbs": "[\"Publishing data\"]", "references": "[{\"href\": \"https://github.com/simonw/csvs-to-sqlite/\", \"label\": \"csvs-to-sqlite\"}, {\"href\": \"https://www.heroku.com/\", \"label\": \"Heroku\"}, {\"href\": \"https://cloud.google.com/\", \"label\": \"Google Cloud\"}]"}
{"id": "contributing:contributing-running-tests", "page": "contributing", "ref": "contributing-running-tests", "title": "Running the tests", "content": "Once you have done this, you can run the Datasette unit tests from inside your  datasette/  directory using  pytest  like so: \n             pytest \n             You can run the tests faster using multiple CPU cores with  pytest-xdist  like this: \n             pytest -n auto -m \"not serial\" \n             -n auto  detects the number of available cores automatically. The  -m \"not serial\"  skips tests that don't work well in a parallel test environment. You can run those tests separately like so: \n             pytest -m \"serial\"", "breadcrumbs": "[\"Contributing\"]", "references": "[{\"href\": \"https://docs.pytest.org/\", \"label\": \"pytest\"}, {\"href\": \"https://pypi.org/project/pytest-xdist/\", \"label\": \"pytest-xdist\"}]"}
{"id": "deploying:deploying-openrc", "page": "deploying", "ref": "deploying-openrc", "title": "Running Datasette using OpenRC", "content": "OpenRC is the service manager on non-systemd Linux distributions like  Alpine Linux  and  Gentoo . \n             Create an init script at  /etc/init.d/datasette  with the following contents: \n             #!/sbin/openrc-run\n\nname=\"datasette\"\ncommand=\"datasette\"\ncommand_args=\"serve -h 0.0.0.0 /path/to/db.db\"\ncommand_background=true\npidfile=\"/run/${RC_SVCNAME}.pid\" \n             You then need to configure the service to run at boot and start it: \n             rc-update add datasette\nrc-service datasette start", "breadcrumbs": "[\"Deploying Datasette\"]", "references": "[{\"href\": \"https://www.alpinelinux.org/\", \"label\": \"Alpine Linux\"}, {\"href\": \"https://www.gentoo.org/\", \"label\": \"Gentoo\"}]"}
{"id": "cli-reference:cli-help-plugins-help", "page": "cli-reference", "ref": "cli-help-plugins-help", "title": "datasette plugins", "content": "Output JSON showing all currently installed plugins, their versions, whether they include static files or templates and which  Plugin hooks  they use. \n             [[[cog\nhelp([\"plugins\", \"--help\"]) \n             ]]] \n             Usage: datasette plugins [OPTIONS]\n\n  List currently installed plugins\n\nOptions:\n  --all                    Include built-in default plugins\n  --plugins-dir DIRECTORY  Path to directory containing custom plugins\n  --help                   Show this message and exit. \n             [[[end]]] \n             Example output: \n             [\n    {\n        \"name\": \"datasette-geojson\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": \"0.3.1\",\n        \"hooks\": [\n            \"register_output_renderer\"\n        ]\n    },\n    {\n        \"name\": \"datasette-geojson-map\",\n        \"static\": true,\n        \"templates\": false,\n        \"version\": \"0.4.0\",\n        \"hooks\": [\n            \"extra_body_script\",\n            \"extra_css_urls\",\n            \"extra_js_urls\"\n        ]\n    },\n    {\n        \"name\": \"datasette-leaflet\",\n        \"static\": true,\n        \"templates\": false,\n        \"version\": \"0.2.2\",\n        \"hooks\": [\n            \"extra_body_script\",\n            \"extra_template_vars\"\n        ]\n    }\n]", "breadcrumbs": "[\"CLI reference\"]", "references": "[]"}
{"id": "cli-reference:cli-help-inspect-help", "page": "cli-reference", "ref": "cli-help-inspect-help", "title": "datasette inspect", "content": "Outputs JSON representing introspected data about one or more SQLite database files. \n             If you are opening an immutable database, you can pass this file to the  --inspect-data  option to improve Datasette's performance by allowing it to skip running row counts against the database when it first starts running: \n             datasette inspect mydatabase.db > inspect-data.json\ndatasette serve -i mydatabase.db --inspect-file inspect-data.json \n             This performance optimization is used automatically by some of the  datasette publish  commands. You are unlikely to need to apply this optimization manually. \n             [[[cog\nhelp([\"inspect\", \"--help\"]) \n             ]]] \n             Usage: datasette inspect [OPTIONS] [FILES]...\n\n  Generate JSON summary of provided database files\n\n  This can then be passed to \"datasette --inspect-file\" to speed up count\n  operations against immutable database files.\n\nOptions:\n  --inspect-file TEXT\n  --load-extension PATH:ENTRYPOINT?\n                                  Path to a SQLite extension to load, and\n                                  optional entrypoint\n  --help                          Show this message and exit. \n             [[[end]]]", "breadcrumbs": "[\"CLI reference\"]", "references": "[]"}
{"id": "cli-reference:cli-help-package-help", "page": "cli-reference", "ref": "cli-help-package-help", "title": "datasette package", "content": "Package SQLite files into a Datasette Docker container, see  datasette package . \n             [[[cog\nhelp([\"package\", \"--help\"]) \n             ]]] \n             Usage: datasette package [OPTIONS] FILES...\n\n  Package SQLite files into a Datasette Docker container\n\nOptions:\n  -t, --tag TEXT            Name for the resulting Docker container, can\n                            optionally use name:tag format\n  -m, --metadata FILENAME   Path to JSON/YAML file containing metadata to\n                            publish\n  --extra-options TEXT      Extra options to pass to datasette serve\n  --branch TEXT             Install datasette from a GitHub branch e.g. main\n  --template-dir DIRECTORY  Path to directory containing custom templates\n  --plugins-dir DIRECTORY   Path to directory containing custom plugins\n  --static MOUNT:DIRECTORY  Serve static files from this directory at /MOUNT/...\n  --install TEXT            Additional packages (e.g. plugins) to install\n  --spatialite              Enable SpatialLite extension\n  --version-note TEXT       Additional note to show on /-/versions\n  --secret TEXT             Secret used for signing secure values, such as\n                            signed cookies\n  -p, --port INTEGER RANGE  Port to run the server on, defaults to 8001\n                            [1<=x<=65535]\n  --title TEXT              Title for metadata\n  --license TEXT            License label for metadata\n  --license_url TEXT        License URL for metadata\n  --source TEXT             Source label for metadata\n  --source_url TEXT         Source URL for metadata\n  --about TEXT              About label for metadata\n  --about_url TEXT          About URL for metadata\n  --help                    Show this message and exit. \n             [[[end]]]", "breadcrumbs": "[\"CLI reference\"]", "references": "[]"}
{"id": "writing_plugins:writing-plugins-packaging", "page": "writing_plugins", "ref": "writing-plugins-packaging", "title": "Packaging a plugin", "content": "Plugins can be packaged using Python setuptools. You can see an example of a packaged plugin at  https://github.com/simonw/datasette-plugin-demos \n             The example consists of two files: a  setup.py  file that defines the plugin: \n             from setuptools import setup\n\nVERSION = \"0.1\"\n\nsetup(\n    name=\"datasette-plugin-demos\",\n    description=\"Examples of plugins for Datasette\",\n    author=\"Simon Willison\",\n    url=\"https://github.com/simonw/datasette-plugin-demos\",\n    license=\"Apache License, Version 2.0\",\n    version=VERSION,\n    py_modules=[\"datasette_plugin_demos\"],\n    entry_points={\n        \"datasette\": [\n            \"plugin_demos = datasette_plugin_demos\"\n        ]\n    },\n    install_requires=[\"datasette\"],\n) \n             And a Python module file,  datasette_plugin_demos.py , that implements the plugin: \n             from datasette import hookimpl\nimport random\n\n\n@hookimpl\ndef prepare_jinja2_environment(env):\n    env.filters[\"uppercase\"] = lambda u: u.upper()\n\n\n@hookimpl\ndef prepare_connection(conn):\n    conn.create_function(\n        \"random_integer\", 2, random.randint\n    ) \n             Having built a plugin in this way you can turn it into an installable package using the following command: \n             python3 setup.py sdist \n             This will create a  .tar.gz  file in the  dist/  directory. \n             You can then install your new plugin into a Datasette virtual environment or Docker container using  pip : \n             pip install datasette-plugin-demos-0.1.tar.gz \n             To learn how to upload your plugin to  PyPI  for use by other people, read the PyPA guide to  Packaging and distributing projects .", "breadcrumbs": "[\"Writing plugins\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette-plugin-demos\", \"label\": \"https://github.com/simonw/datasette-plugin-demos\"}, {\"href\": \"https://pypi.org/\", \"label\": \"PyPI\"}, {\"href\": \"https://packaging.python.org/tutorials/distributing-packages/\", \"label\": \"Packaging and distributing projects\"}]"}
{"id": "plugins:plugins-configuration", "page": "plugins", "ref": "plugins-configuration", "title": "Plugin configuration", "content": "Plugins can have their own configuration, embedded in a  Metadata  file. Configuration options for plugins live within a  \"plugins\"  key in that file, which can be included at the root, database or table level. \n             Here is an example of some plugin configuration for a specific table: \n             {\n    \"databases\": {\n        \"sf-trees\": {\n            \"tables\": {\n                \"Street_Tree_List\": {\n                    \"plugins\": {\n                        \"datasette-cluster-map\": {\n                            \"latitude_column\": \"lat\",\n                            \"longitude_column\": \"lng\"\n                        }\n                    }\n                }\n            }\n        }\n    }\n} \n             This tells the  datasette-cluster-map  column which latitude and longitude columns should be used for a table called  Street_Tree_List  inside a database file called  sf-trees.db .", "breadcrumbs": "[\"Plugins\"]", "references": "[]"}
{"id": "internals:internals-datasette-client", "page": "internals", "ref": "internals-datasette-client", "title": "datasette.client", "content": "Plugins can make internal simulated HTTP requests to the Datasette instance within which they are running. This ensures that all of Datasette's external JSON APIs are also available to plugins, while avoiding the overhead of making an external HTTP call to access those APIs. \n                 The  datasette.client  object is a wrapper around the  HTTPX Python library , providing an async-friendly API that is similar to the widely used  Requests library . \n                 It offers the following methods: \n                 \n                     \n                         await datasette.client.get(path, **kwargs)  - returns HTTPX Response \n                         \n                             Execute an internal GET request against that path. \n                         \n                     \n                     \n                         await datasette.client.post(path, **kwargs)  - returns HTTPX Response \n                         \n                             Execute an internal POST request. Use  data={\"name\": \"value\"}  to pass form parameters. \n                         \n                     \n                     \n                         await datasette.client.options(path, **kwargs)  - returns HTTPX Response \n                         \n                             Execute an internal OPTIONS request. \n                         \n                     \n                     \n                         await datasette.client.head(path, **kwargs)  - returns HTTPX Response \n                         \n                             Execute an internal HEAD request. \n                         \n                     \n                     \n                         await datasette.client.put(path, **kwargs)  - returns HTTPX Response \n                         \n                             Execute an internal PUT request. \n                         \n                     \n                     \n                         await datasette.client.patch(path, **kwargs)  - returns HTTPX Response \n                         \n                             Execute an internal PATCH request. \n                         \n                     \n                     \n                         await datasette.client.delete(path, **kwargs)  - returns HTTPX Response \n                         \n                             Execute an internal DELETE request. \n                         \n                     \n                     \n                         await datasette.client.request(method, path, **kwargs)  - returns HTTPX Response \n                         \n                             Execute an internal request with the given HTTP method against that path. \n                         \n                     \n                 \n                 These methods can be used with  datasette.urls  - for example: \n                 table_json = (\n    await datasette.client.get(\n        datasette.urls.table(\n            \"fixtures\", \"facetable\", format=\"json\"\n        )\n    )\n).json() \n                 datasette.client  methods automatically take the current  base_url  setting into account, whether or not you use the  datasette.urls  family of methods to construct the path. \n                 For documentation on available  **kwargs  options and the shape of the HTTPX Response object refer to the  HTTPX Async documentation .", "breadcrumbs": "[\"Internals for plugins\", \"Datasette class\"]", "references": "[{\"href\": \"https://www.python-httpx.org/\", \"label\": \"HTTPX Python library\"}, {\"href\": \"https://requests.readthedocs.io/\", \"label\": \"Requests library\"}, {\"href\": \"https://www.python-httpx.org/async/\", \"label\": \"HTTPX Async documentation\"}]"}
{"id": "changelog:register-routes-plugin-hooks", "page": "changelog", "ref": "register-routes-plugin-hooks", "title": "register_routes() plugin hooks", "content": "Plugins can now register new views and routes via the  register_routes(datasette)  plugin hook ( #819 ). View functions can be defined that accept any of the current  datasette  object, the current  request , or the ASGI  scope ,  send  and  receive  objects.", "breadcrumbs": "[\"Changelog\", \"0.44 (2020-06-11)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/819\", \"label\": \"#819\"}]"}
{"id": "changelog:cookie-methods", "page": "changelog", "ref": "cookie-methods", "title": "Cookie methods", "content": "Plugins can now use the new  response.set_cookie()  method to set cookies. \n                 A new  request.cookies  method on the :ref:internals_request` can be used to read incoming cookies.", "breadcrumbs": "[\"Changelog\", \"0.44 (2020-06-11)\"]", "references": "[]"}
{"id": "changelog:secret-plugin-configuration-options", "page": "changelog", "ref": "secret-plugin-configuration-options", "title": "Secret plugin configuration options", "content": "Plugins like  datasette-auth-github  need a safe way to set secret configuration options. Since the default mechanism for configuring plugins exposes those settings in  /-/metadata  a new mechanism was needed.  Secret configuration values  describes how plugins can now specify that their settings should be read from a file or an environment variable: \n                 {\n    \"plugins\": {\n        \"datasette-auth-github\": {\n            \"client_secret\": {\n                \"$env\": \"GITHUB_CLIENT_SECRET\"\n            }\n        }\n    }\n} \n                 These plugin secrets can be set directly using  datasette publish . See  Custom metadata and plugins  for details. ( #538  and  #543 )", "breadcrumbs": "[\"Changelog\", \"0.29 (2019-07-07)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette-auth-github\", \"label\": \"datasette-auth-github\"}, {\"href\": \"https://github.com/simonw/datasette/issues/538\", \"label\": \"#538\"}, {\"href\": \"https://github.com/simonw/datasette/issues/543\", \"label\": \"#543\"}]"}
{"id": "changelog:id66", "page": "changelog", "ref": "id66", "title": "0.37 (2020-02-25)", "content": "Plugins now have a supported mechanism for writing to a database, using the new  .execute_write()  and  .execute_write_fn()  methods.  Documentation . ( #682 ) \n                 \n                 \n                     Immutable databases that have had their rows counted using the  inspect  command now use the calculated count more effectively - thanks, Kevin Keogh. ( #666 ) \n                 \n                 \n                     --reload  no longer restarts the server if a database file is modified, unless that database was opened immutable mode with  -i . ( #494 ) \n                 \n                 \n                     New  ?_searchmode=raw  option turns off escaping for FTS queries in  ?_search=  allowing full use of SQLite's  FTS5 query syntax . ( #676 )", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/682\", \"label\": \"#682\"}, {\"href\": \"https://github.com/simonw/datasette/pull/666\", \"label\": \"#666\"}, {\"href\": \"https://github.com/simonw/datasette/issues/494\", \"label\": \"#494\"}, {\"href\": \"https://www.sqlite.org/fts5.html#full_text_query_syntax\", \"label\": \"FTS5 query syntax\"}, {\"href\": \"https://github.com/simonw/datasette/issues/676\", \"label\": \"#676\"}]"}
{"id": "writing_plugins:writing-plugins-cookiecutter", "page": "writing_plugins", "ref": "writing-plugins-cookiecutter", "title": "Starting an installable plugin using cookiecutter", "content": "Plugins that can be installed should be written as Python packages using a  setup.py  file. \n             The quickest way to start writing one an installable plugin is to use the  datasette-plugin  cookiecutter template. This creates a new plugin structure for you complete with an example test and GitHub Actions workflows for testing and publishing your plugin. \n             Install cookiecutter  and then run this command to start building a plugin using the template: \n             cookiecutter gh:simonw/datasette-plugin \n             Read  a cookiecutter template for writing Datasette plugins  for more information about this template.", "breadcrumbs": "[\"Writing plugins\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette-plugin\", \"label\": \"datasette-plugin\"}, {\"href\": \"https://cookiecutter.readthedocs.io/en/stable/installation.html\", \"label\": \"Install cookiecutter\"}, {\"href\": \"https://simonwillison.net/2020/Jun/20/cookiecutter-plugins/\", \"label\": \"a cookiecutter template for writing Datasette plugins\"}]"}
{"id": "writing_plugins:writing-plugins-building-urls", "page": "writing_plugins", "ref": "writing-plugins-building-urls", "title": "Building URLs within plugins", "content": "Plugins that define their own custom user interface elements may need to link to other pages within Datasette. \n             This can be a bit tricky if the Datasette instance is using the  base_url  configuration setting to run behind a proxy, since that can cause Datasette's URLs to include an additional prefix. \n             The  datasette.urls  object provides internal methods for correctly generating URLs to different pages within Datasette, taking any  base_url  configuration into account. \n             This object is exposed in templates as the  urls  variable, which can be used like this: \n             Back to the <a href=\"{{ urls.instance() }}\">Homepage</a> \n             See  datasette.urls  for full details on this object.", "breadcrumbs": "[\"Writing plugins\"]", "references": "[]"}
{"id": "authentication:authentication-actor-matches-allow", "page": "authentication", "ref": "authentication-actor-matches-allow", "title": "actor_matches_allow()", "content": "Plugins that wish to implement this same  \"allow\"  block permissions scheme can take advantage of the  datasette.utils.actor_matches_allow(actor, allow)  function: \n             from datasette.utils import actor_matches_allow\n\nactor_matches_allow({\"id\": \"root\"}, {\"id\": \"*\"})\n# returns True \n             The currently authenticated actor is made available to plugins as  request.actor .", "breadcrumbs": "[\"Authentication and permissions\"]", "references": "[]"}
{"id": "changelog:authentication", "page": "changelog", "ref": "authentication", "title": "Authentication", "content": "Prior to this release the Datasette ecosystem has treated authentication as exclusively the realm of plugins, most notably through  datasette-auth-github . \n                 0.44 introduces  Authentication and permissions  as core Datasette concepts ( #699 ). This enables different plugins to share responsibility for authenticating requests - you might have one plugin that handles user accounts and another one that allows automated access via API keys, for example. \n                 You'll need to install plugins if you want full user accounts, but default Datasette can now authenticate a single root user with the new  --root  command-line option, which outputs a one-time use URL to  authenticate as a root actor  ( #784 ): \n                 $ datasette fixtures.db --root\nhttp://127.0.0.1:8001/-/auth-token?token=5b632f8cd44b868df625f5a6e2185d88eea5b22237fd3cc8773f107cc4fd6477\nINFO:     Started server process [14973]\nINFO:     Waiting for application startup.\nINFO:     Application startup complete.\nINFO:     Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit) \n                 Plugins can implement new ways of authenticating users using the new  actor_from_request(datasette, request)  hook.", "breadcrumbs": "[\"Changelog\", \"0.44 (2020-06-11)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette-auth-github\", \"label\": \"datasette-auth-github\"}, {\"href\": \"https://github.com/simonw/datasette/issues/699\", \"label\": \"#699\"}, {\"href\": \"https://github.com/simonw/datasette/issues/784\", \"label\": \"#784\"}]"}
{"id": "internals:datasette-databases", "page": "internals", "ref": "datasette-databases", "title": ".databases", "content": "Property exposing a  collections.OrderedDict  of databases currently connected to Datasette. \n                 The dictionary keys are the name of the database that is used in the URL - e.g.  /fixtures  would have a key of  \"fixtures\" . The values are  Database class  instances. \n                 All databases are listed, irrespective of user permissions. This means that the  _internal  database will always be listed here.", "breadcrumbs": "[\"Internals for plugins\", \"Datasette class\"]", "references": "[]"}
{"id": "testing_plugins:testing-plugins-fixtures", "page": "testing_plugins", "ref": "testing-plugins-fixtures", "title": "Using pytest fixtures", "content": "Pytest fixtures  can be used to create initial testable objects which can then be used by multiple tests. \n             A common pattern for Datasette plugins is to create a fixture which sets up a temporary test database and wraps it in a Datasette instance. \n             Here's an example that uses the  sqlite-utils library  to populate a temporary test database. It also sets the title of that table using a simulated  metadata.json  configuration: \n             from datasette.app import Datasette\nimport pytest\nimport sqlite_utils\n\n\n@pytest.fixture(scope=\"session\")\ndef datasette(tmp_path_factory):\n    db_directory = tmp_path_factory.mktemp(\"dbs\")\n    db_path = db_directory / \"test.db\"\n    db = sqlite_utils.Database(db_path)\n    db[\"dogs\"].insert_all(\n        [\n            {\"id\": 1, \"name\": \"Cleo\", \"age\": 5},\n            {\"id\": 2, \"name\": \"Pancakes\", \"age\": 4},\n        ],\n        pk=\"id\",\n    )\n    datasette = Datasette(\n        [db_path],\n        metadata={\n            \"databases\": {\n                \"test\": {\n                    \"tables\": {\n                        \"dogs\": {\"title\": \"Some dogs\"}\n                    }\n                }\n            }\n        },\n    )\n    return datasette\n\n\n@pytest.mark.asyncio\nasync def test_example_table_json(datasette):\n    response = await datasette.client.get(\n        \"/test/dogs.json?_shape=array\"\n    )\n    assert response.status_code == 200\n    assert response.json() == [\n        {\"id\": 1, \"name\": \"Cleo\", \"age\": 5},\n        {\"id\": 2, \"name\": \"Pancakes\", \"age\": 4},\n    ]\n\n\n@pytest.mark.asyncio\nasync def test_example_table_html(datasette):\n    response = await datasette.client.get(\"/test/dogs\")\n    assert \">Some dogs</h1>\" in response.text \n             Here the  datasette()  function defines the fixture, which is than automatically passed to the two test functions based on pytest automatically matching their  datasette  function parameters. \n             The  @pytest.fixture(scope=\"session\")  line here ensures the fixture is reused for the full  pytest  execution session. This means that the temporary database file will be created once and reused for each test. \n             If you want to create that test database repeatedly for every individual test function, write the fixture function like this instead. You may want to do this if your plugin modifies the database contents in some way: \n             @pytest.fixture\ndef datasette(tmp_path_factory):\n    # This fixture will be executed repeatedly for every test\n    ...", "breadcrumbs": "[\"Testing plugins\"]", "references": "[{\"href\": \"https://docs.pytest.org/en/stable/fixture.html\", \"label\": \"Pytest fixtures\"}, {\"href\": \"https://sqlite-utils.datasette.io/en/stable/python-api.html\", \"label\": \"sqlite-utils library\"}]"}
{"id": "changelog:id151", "page": "changelog", "ref": "id151", "title": "0.17 (2018-04-13)", "content": "Release 0.17 to fix issues with PyPI", "breadcrumbs": "[\"Changelog\"]", "references": "[]"}
{"id": "plugin_hooks:plugin-register-facet-classes", "page": "plugin_hooks", "ref": "plugin-register-facet-classes", "title": "register_facet_classes()", "content": "Return a list of additional Facet subclasses to be registered. \n             \n                 The design of this plugin hook is unstable and may change. See  issue 830 . \n             \n             Each Facet subclass implements a new type of facet operation. The class should look like this: \n             class SpecialFacet(Facet):\n    # This key must be unique across all facet classes:\n    type = \"special\"\n\n    async def suggest(self):\n        # Use self.sql and self.params to suggest some facets\n        suggested_facets = []\n        suggested_facets.append(\n            {\n                \"name\": column,  # Or other unique name\n                # Construct the URL that will enable this facet:\n                \"toggle_url\": self.ds.absolute_url(\n                    self.request,\n                    path_with_added_args(\n                        self.request, {\"_facet\": column}\n                    ),\n                ),\n            }\n        )\n        return suggested_facets\n\n    async def facet_results(self):\n        # This should execute the facet operation and return results, again\n        # using self.sql and self.params as the starting point\n        facet_results = []\n        facets_timed_out = []\n        facet_size = self.get_facet_size()\n        # Do some calculations here...\n        for column in columns_selected_for_facet:\n            try:\n                facet_results_values = []\n                # More calculations...\n                facet_results_values.append(\n                    {\n                        \"value\": value,\n                        \"label\": label,\n                        \"count\": count,\n                        \"toggle_url\": self.ds.absolute_url(\n                            self.request, toggle_path\n                        ),\n                        \"selected\": selected,\n                    }\n                )\n                facet_results.append(\n                    {\n                        \"name\": column,\n                        \"results\": facet_results_values,\n                        \"truncated\": len(facet_rows_results)\n                        > facet_size,\n                    }\n                )\n            except QueryInterrupted:\n                facets_timed_out.append(column)\n\n        return facet_results, facets_timed_out \n             See  datasette/facets.py  for examples of how these classes can work. \n             The plugin hook can then be used to register the new facet class like this: \n             @hookimpl\ndef register_facet_classes():\n    return [SpecialFacet]", "breadcrumbs": "[\"Plugin hooks\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/830\", \"label\": \"issue 830\"}, {\"href\": \"https://github.com/simonw/datasette/blob/main/datasette/facets.py\", \"label\": \"datasette/facets.py\"}]"}
{"id": "plugin_hooks:plugin-asgi-wrapper", "page": "plugin_hooks", "ref": "plugin-asgi-wrapper", "title": "asgi_wrapper(datasette)", "content": "Return an  ASGI  middleware wrapper function that will be applied to the Datasette ASGI application. \n             This is a very powerful hook. You can use it to manipulate the entire Datasette response, or even to configure new URL routes that will be handled by your own custom code. \n             You can write your ASGI code directly against the low-level specification, or you can use the middleware utilities provided by an ASGI framework such as  Starlette . \n             This example plugin adds a  x-databases  HTTP header listing the currently attached databases: \n             from datasette import hookimpl\nfrom functools import wraps\n\n\n@hookimpl\ndef asgi_wrapper(datasette):\n    def wrap_with_databases_header(app):\n        @wraps(app)\n        async def add_x_databases_header(\n            scope, receive, send\n        ):\n            async def wrapped_send(event):\n                if event[\"type\"] == \"http.response.start\":\n                    original_headers = (\n                        event.get(\"headers\") or []\n                    )\n                    event = {\n                        \"type\": event[\"type\"],\n                        \"status\": event[\"status\"],\n                        \"headers\": original_headers\n                        + [\n                            [\n                                b\"x-databases\",\n                                \", \".join(\n                                    datasette.databases.keys()\n                                ).encode(\"utf-8\"),\n                            ]\n                        ],\n                    }\n                await send(event)\n\n            await app(scope, receive, wrapped_send)\n\n        return add_x_databases_header\n\n    return wrap_with_databases_header \n             Examples:  datasette-cors ,  datasette-pyinstrument ,  datasette-total-page-time", "breadcrumbs": "[\"Plugin hooks\"]", "references": "[{\"href\": \"https://asgi.readthedocs.io/\", \"label\": \"ASGI\"}, {\"href\": \"https://www.starlette.io/middleware/\", \"label\": \"Starlette\"}, {\"href\": \"https://datasette.io/plugins/datasette-cors\", \"label\": \"datasette-cors\"}, {\"href\": \"https://datasette.io/plugins/datasette-pyinstrument\", \"label\": \"datasette-pyinstrument\"}, {\"href\": \"https://datasette.io/plugins/datasette-total-page-time\", \"label\": \"datasette-total-page-time\"}]"}
{"id": "cli-reference:cli-help-help", "page": "cli-reference", "ref": "cli-help-help", "title": "datasette --help", "content": "Running  datasette --help  shows a list of all of the available commands. \n             [[[cog\nhelp([\"--help\"]) \n             ]]] \n             Usage: datasette [OPTIONS] COMMAND [ARGS]...\n\n  Datasette is an open source multi-tool for exploring and publishing data\n\n  About Datasette: https://datasette.io/\n  Full documentation: https://docs.datasette.io/\n\nOptions:\n  --version  Show the version and exit.\n  --help     Show this message and exit.\n\nCommands:\n  serve*     Serve up specified SQLite database files with a web UI\n  inspect    Generate JSON summary of provided database files\n  install    Install plugins and packages from PyPI into the same...\n  package    Package SQLite files into a Datasette Docker container\n  plugins    List currently installed plugins\n  publish    Publish specified SQLite database files to the internet along...\n  uninstall  Uninstall plugins and Python packages from the Datasette... \n             [[[end]]] \n             Additional commands added by plugins that use the  register_commands(cli)  hook will be listed here as well.", "breadcrumbs": "[\"CLI reference\"]", "references": "[]"}
{"id": "internals:internals-tracer", "page": "internals", "ref": "internals-tracer", "title": "datasette.tracer", "content": "Running Datasette with  --setting trace_debug 1  enables trace debug output, which can then be viewed by adding  ?_trace=1  to the query string for any page. \n             You can see an example of this at the bottom of  latest.datasette.io/fixtures/facetable?_trace=1 . The JSON output shows full details of every SQL query that was executed to generate the page. \n             The  datasette-pretty-traces  plugin can be installed to provide a more readable display of this information. You can see  a demo of that here . \n             You can add your own custom traces to the JSON output using the  trace()  context manager. This takes a string that identifies the type of trace being recorded, and records any keyword arguments as additional JSON keys on the resulting trace object. \n             The start and end time, duration and a traceback of where the trace was executed will be automatically attached to the JSON object. \n             This example uses trace to record the start, end and duration of any HTTP GET requests made using the function: \n             from datasette.tracer import trace\nimport httpx\n\n\nasync def fetch_url(url):\n    with trace(\"fetch-url\", url=url):\n        async with httpx.AsyncClient() as client:\n            return await client.get(url)", "breadcrumbs": "[\"Internals for plugins\"]", "references": "[{\"href\": \"https://latest.datasette.io/fixtures/facetable?_trace=1\", \"label\": \"latest.datasette.io/fixtures/facetable?_trace=1\"}, {\"href\": \"https://datasette.io/plugins/datasette-pretty-traces\", \"label\": \"datasette-pretty-traces\"}, {\"href\": \"https://latest-with-plugins.datasette.io/github/commits?_trace=1\", \"label\": \"a demo of that here\"}]"}
{"id": "changelog:v0-28-register-output-renderer", "page": "changelog", "ref": "v0-28-register-output-renderer", "title": "register_output_renderer plugins", "content": "Russ Garrett implemented a new Datasette plugin hook called  register_output_renderer  ( #441 ) which allows plugins to create additional output renderers in addition to Datasette's default  .json  and  .csv . \n                 Russ's in-development  datasette-geo  plugin includes  an example  of this hook being used to output  .geojson  automatically converted from SpatiaLite.", "breadcrumbs": "[\"Changelog\", \"0.28 (2019-05-19)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/pull/441\", \"label\": \"#441\"}, {\"href\": \"https://github.com/russss/datasette-geo\", \"label\": \"datasette-geo\"}, {\"href\": \"https://github.com/russss/datasette-geo/blob/d4cecc020848bbde91e9e17bf352f7c70bc3dccf/datasette_plugin_geo/geojson.py\", \"label\": \"an example\"}]"}
{"id": "full_text_search:full-text-search-advanced-queries", "page": "full_text_search", "ref": "full-text-search-advanced-queries", "title": "Advanced SQLite search queries", "content": "SQLite full-text search includes support for  a variety of advanced queries , including  AND ,  OR ,  NOT  and  NEAR . \n             By default Datasette disables these features to ensure they do not cause errors or confusion for users who are not aware of them. You can disable this escaping and use the advanced queries by adding  &_searchmode=raw  to the table page query string. \n             If you want to enable these operators by default for a specific table, you can do so by adding  \"searchmode\": \"raw\"  to the metadata configuration for that table, see  Configuring full-text search for a table or view . \n             If that option has been specified in the table metadata but you want to over-ride it and return to the default behavior you can append  &_searchmode=escaped  to the query string.", "breadcrumbs": "[\"Full-text search\"]", "references": "[{\"href\": \"https://www.sqlite.org/fts5.html#full_text_query_syntax\", \"label\": \"a variety of advanced queries\"}]"}
{"id": "sql_queries:id3", "page": "sql_queries", "ref": "id3", "title": "Cross-database queries", "content": "SQLite has the ability to run queries that join across multiple databases. Up to ten databases can be attached to a single SQLite connection and queried together. \n             Datasette can execute joins across multiple databases if it is started with the  --crossdb  option: \n             datasette fixtures.db extra_database.db --crossdb \n             If it is started in this way, the  /_memory  page can be used to execute queries that join across multiple databases. \n             References to tables in attached databases should be preceded by the database name and a period. \n             For example, this query will show a list of tables across both of the above databases: \n             select\n  'fixtures' as database, *\nfrom\n  [fixtures].sqlite_master\nunion\nselect\n  'extra_database' as database, *\nfrom\n  [extra_database].sqlite_master \n             Try that out here .", "breadcrumbs": "[\"Running SQL queries\"]", "references": "[{\"href\": \"https://latest.datasette.io/_memory?sql=select%0D%0A++%27fixtures%27+as+database%2C+*%0D%0Afrom%0D%0A++%5Bfixtures%5D.sqlite_master%0D%0Aunion%0D%0Aselect%0D%0A++%27extra_database%27+as+database%2C+*%0D%0Afrom%0D%0A++%5Bextra_database%5D.sqlite_master\", \"label\": \"Try that out here\"}]"}
{"id": "full_text_search:id1", "page": "full_text_search", "ref": "id1", "title": "Full-text search", "content": "SQLite includes  a powerful mechanism for enabling full-text search  against SQLite records. Datasette can detect if a table has had full-text search configured for it in the underlying database and display a search interface for filtering that table. \n         Here's  an example search : \n         \n         Datasette automatically detects which tables have been configured for full-text search.", "breadcrumbs": "[]", "references": "[{\"href\": \"https://www.sqlite.org/fts3.html\", \"label\": \"a powerful mechanism for enabling full-text search\"}, {\"href\": \"https://register-of-members-interests.datasettes.com/regmem/items?_search=hamper&_sort_desc=date\", \"label\": \"an example search\"}]"}
{"id": "internals:database-execute-write", "page": "internals", "ref": "database-execute-write", "title": "await db.execute_write(sql, params=None, block=True)", "content": "SQLite only allows one database connection to write at a time. Datasette handles this for you by maintaining a queue of writes to be executed against a given database. Plugins can submit write operations to this queue and they will be executed in the order in which they are received. \n                 This method can be used to queue up a non-SELECT SQL query to be executed against a single write connection to the database. \n                 You can pass additional SQL parameters as a tuple or dictionary. \n                 The method will block until the operation is completed, and the return value will be the return from calling  conn.execute(...)  using the underlying  sqlite3  Python library. \n                 If you pass  block=False  this behaviour changes to \"fire and forget\" - queries will be added to the write queue and executed in a separate thread while your code can continue to do other things. The method will return a UUID representing the queued task.", "breadcrumbs": "[\"Internals for plugins\", \"Database class\"]", "references": "[]"}
{"id": "installation:installation-extensions", "page": "installation", "ref": "installation-extensions", "title": "A note about extensions", "content": "SQLite supports extensions, such as  SpatiaLite  for geospatial operations. \n             These can be loaded using the  --load-extension  argument, like so: \n             datasette --load-extension=/usr/local/lib/mod_spatialite.dylib \n             Some Python installations do not include support for SQLite extensions. If this is the case you will see the following error when you attempt to load an extension: \n             \n                 Your Python installation does not have the ability to load SQLite extensions. \n             \n             In some cases you may see the following error message instead: \n             AttributeError: 'sqlite3.Connection' object has no attribute 'enable_load_extension' \n             On macOS the easiest fix for this is to install Datasette using Homebrew: \n             brew install datasette \n             Use  which datasette  to confirm that  datasette  will run that version. The output should look something like this: \n             /usr/local/opt/datasette/bin/datasette \n             If you get a different location here such as  /Library/Frameworks/Python.framework/Versions/3.10/bin/datasette  you can run the following command to cause  datasette  to execute the Homebrew version instead: \n             alias datasette=$(echo $(brew --prefix datasette)/bin/datasette) \n             You can undo this operation using: \n             unalias datasette \n             If you need to run SQLite with extension support for other Python code, you can do so by install Python itself using Homebrew: \n             brew install python \n             Then executing Python using: \n             /usr/local/opt/python@3/libexec/bin/python \n             A more convenient way to work with this version of Python may be to use it to create a virtual environment: \n             /usr/local/opt/python@3/libexec/bin/python -m venv datasette-venv \n             Then activate it like this: \n             source datasette-venv/bin/activate \n             Now running  python  and  pip  will work against a version of Python 3 that includes support for SQLite extensions: \n             pip install datasette\nwhich datasette\ndatasette --version", "breadcrumbs": "[\"Installation\"]", "references": "[]"}
{"id": "binary_data:binary", "page": "binary_data", "ref": "binary", "title": "Binary data", "content": "SQLite tables can contain binary data in  BLOB  columns. \n         Datasette includes special handling for these binary values. The Datasette interface detects binary values and provides a link to download their content, for example on  https://latest.datasette.io/fixtures/binary_data \n         \n         Binary data is represented in  .json  exports using Base64 encoding. \n         https://latest.datasette.io/fixtures/binary_data.json?_shape=array \n         [\n    {\n        \"rowid\": 1,\n        \"data\": {\n            \"$base64\": true,\n            \"encoded\": \"FRwCx60F/g==\"\n        }\n    },\n    {\n        \"rowid\": 2,\n        \"data\": {\n            \"$base64\": true,\n            \"encoded\": \"FRwDx60F/g==\"\n        }\n    },\n    {\n        \"rowid\": 3,\n        \"data\": null\n    }\n]", "breadcrumbs": "[]", "references": "[{\"href\": \"https://latest.datasette.io/fixtures/binary_data\", \"label\": \"https://latest.datasette.io/fixtures/binary_data\"}, {\"href\": \"https://latest.datasette.io/fixtures/binary_data.json?_shape=array\", \"label\": \"https://latest.datasette.io/fixtures/binary_data.json?_shape=array\"}]"}
{"id": "changelog:binary-data", "page": "changelog", "ref": "binary-data", "title": "Binary data", "content": "SQLite tables can contain binary data in  BLOB  columns. Datasette now provides links for users to download this data directly from Datasette, and uses those links to make binary data available from CSV exports. See  Binary data  for more details. ( #1036  and  #1034 ).", "breadcrumbs": "[\"Changelog\", \"0.51 (2020-10-31)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/1036\", \"label\": \"#1036\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1034\", \"label\": \"#1034\"}]"}
{"id": "changelog:id180", "page": "changelog", "ref": "id180", "title": "0.13 (2017-11-24)", "content": "Search now applies to current filters. \n                     Combined search into the same form as filters. \n                     Closes  #133 \n                 \n                 \n                     Much tidier design for table view header. \n                     Closes  #147 \n                 \n                 \n                     Added  ?column__not=blah  filter. \n                     Closes  #148 \n                 \n                 \n                     Row page now resolves foreign keys. \n                     Closes  #132 \n                 \n                 \n                     Further tweaks to select/input filter styling. \n                     Refs  #86  - thanks for the help, @natbat! \n                 \n                 \n                     Show linked foreign key in table cells. \n                 \n                 \n                     Added UI for editing table filters. \n                     Refs  #86 \n                 \n                 \n                     Hide FTS-created tables on index pages. \n                     Closes  #129 \n                 \n                 \n                     Add publish to heroku support [Jacob Kaplan-Moss] \n                     datasette publish heroku mydb.db \n                     Pull request  #104 \n                 \n                 \n                     Initial implementation of  ?_group_count=column . \n                     URL shortcut for counting rows grouped by one or more columns. \n                     ?_group_count=column1&_group_count=column2  works as well. \n                     SQL generated looks like this: \n                     select \"qSpecies\", count(*) as \"count\"\nfrom Street_Tree_List\ngroup by \"qSpecies\"\norder by \"count\" desc limit 100 \n                     Or for two columns like this: \n                     select \"qSpecies\", \"qSiteInfo\", count(*) as \"count\"\nfrom Street_Tree_List\ngroup by \"qSpecies\", \"qSiteInfo\"\norder by \"count\" desc limit 100 \n                     Refs  #44 \n                 \n                 \n                     Added  --build=master  option to datasette publish and package. \n                     The  datasette publish  and  datasette package  commands both now accept an\n                        optional  --build  argument. If provided, this can be used to specify a branch\n                        published to GitHub that should be built into the container. \n                     This makes it easier to test code that has not yet been officially released to\n                        PyPI, e.g.: \n                     datasette publish now mydb.db --branch=master \n                 \n                 \n                     Implemented  ?_search=XXX  + UI if a FTS table is detected. \n                     Closes  #131 \n                 \n                 \n                     Added  datasette --version  support. \n                 \n                 \n                     Table views now show expanded foreign key references, if possible. \n                     If a table has foreign key columns, and those foreign key tables have\n                         label_columns , the TableView will now query those other tables for the\n                        corresponding values and display those values as links in the corresponding\n                        table cells. \n                     label_columns are currently detected by the  inspect()  function, which looks\n                        for any table that has just two columns - an ID column and one other - and\n                        sets the  label_column  to be that second non-ID column. \n                 \n                 \n                     Don't prevent tabbing to \"Run SQL\" button ( #117 ) [Robert Gieseke] \n                     See comment in  #115 \n                 \n                 \n                     Add keyboard shortcut to execute SQL query ( #115 ) [Robert Gieseke] \n                 \n                 \n                     Allow  --load-extension  to be set via environment variable. \n                 \n                 \n                     Add support for  ?field__isnull=1  ( #107 ) [Ray N] \n                 \n                 \n                     Add spatialite, switch to debian and local build ( #114 ) [Ariel N\u00fa\u00f1ez] \n                 \n                 \n                     Added  --load-extension  argument to datasette serve. \n                     Allows loading of SQLite extensions. Refs  #110 .", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/133\", \"label\": \"#133\"}, {\"href\": \"https://github.com/simonw/datasette/issues/147\", \"label\": \"#147\"}, {\"href\": \"https://github.com/simonw/datasette/issues/148\", \"label\": \"#148\"}, {\"href\": \"https://github.com/simonw/datasette/issues/132\", \"label\": \"#132\"}, {\"href\": \"https://github.com/simonw/datasette/issues/86\", \"label\": \"#86\"}, {\"href\": \"https://github.com/simonw/datasette/issues/86\", \"label\": \"#86\"}, {\"href\": \"https://github.com/simonw/datasette/issues/129\", \"label\": \"#129\"}, {\"href\": \"https://github.com/simonw/datasette/issues/104\", \"label\": \"#104\"}, {\"href\": \"https://github.com/simonw/datasette/issues/44\", \"label\": \"#44\"}, {\"href\": \"https://github.com/simonw/datasette/issues/131\", \"label\": \"#131\"}, {\"href\": \"https://github.com/simonw/datasette/issues/117\", \"label\": \"#117\"}, {\"href\": \"https://github.com/simonw/datasette/issues/115\", \"label\": \"#115\"}, {\"href\": \"https://github.com/simonw/datasette/issues/115\", \"label\": \"#115\"}, {\"href\": \"https://github.com/simonw/datasette/issues/107\", \"label\": \"#107\"}, {\"href\": \"https://github.com/simonw/datasette/issues/114\", \"label\": \"#114\"}, {\"href\": \"https://github.com/simonw/datasette/issues/110\", \"label\": \"#110\"}]"}
{"id": "changelog:id12", "page": "changelog", "ref": "id12", "title": "0.63 (2022-10-27)", "content": "See  Datasette 0.63: The annotated release notes  for more background on the changes in this release.", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://simonwillison.net/2022/Oct/27/datasette-0-63/\", \"label\": \"Datasette 0.63: The annotated release notes\"}]"}
{"id": "cli-reference:cli-help-publish-cloudrun-help", "page": "cli-reference", "ref": "cli-help-publish-cloudrun-help", "title": "datasette publish cloudrun", "content": "See  Publishing to Google Cloud Run . \n             [[[cog\nhelp([\"publish\", \"cloudrun\", \"--help\"]) \n             ]]] \n             Usage: datasette publish cloudrun [OPTIONS] [FILES]...\n\n  Publish databases to Datasette running on Cloud Run\n\nOptions:\n  -m, --metadata FILENAME         Path to JSON/YAML file containing metadata to\n                                  publish\n  --extra-options TEXT            Extra options to pass to datasette serve\n  --branch TEXT                   Install datasette from a GitHub branch e.g.\n                                  main\n  --template-dir DIRECTORY        Path to directory containing custom templates\n  --plugins-dir DIRECTORY         Path to directory containing custom plugins\n  --static MOUNT:DIRECTORY        Serve static files from this directory at\n                                  /MOUNT/...\n  --install TEXT                  Additional packages (e.g. plugins) to install\n  --plugin-secret <TEXT TEXT TEXT>...\n                                  Secrets to pass to plugins, e.g. --plugin-\n                                  secret datasette-auth-github client_id xxx\n  --version-note TEXT             Additional note to show on /-/versions\n  --secret TEXT                   Secret used for signing secure values, such as\n                                  signed cookies\n  --title TEXT                    Title for metadata\n  --license TEXT                  License label for metadata\n  --license_url TEXT              License URL for metadata\n  --source TEXT                   Source label for metadata\n  --source_url TEXT               Source URL for metadata\n  --about TEXT                    About label for metadata\n  --about_url TEXT                About URL for metadata\n  -n, --name TEXT                 Application name to use when building\n  --service TEXT                  Cloud Run service to deploy (or over-write)\n  --spatialite                    Enable SpatialLite extension\n  --show-files                    Output the generated Dockerfile and\n                                  metadata.json\n  --memory TEXT                   Memory to allocate in Cloud Run, e.g. 1Gi\n  --cpu [1|2|4]                   Number of vCPUs to allocate in Cloud Run\n  --timeout INTEGER               Build timeout in seconds\n  --apt-get-install TEXT          Additional packages to apt-get install\n  --max-instances INTEGER         Maximum Cloud Run instances\n  --min-instances INTEGER         Minimum Cloud Run instances\n  --help                          Show this message and exit. \n             [[[end]]]", "breadcrumbs": "[\"CLI reference\"]", "references": "[]"}
{"id": "cli-reference:cli-help-publish-heroku-help", "page": "cli-reference", "ref": "cli-help-publish-heroku-help", "title": "datasette publish heroku", "content": "See  Publishing to Heroku . \n             [[[cog\nhelp([\"publish\", \"heroku\", \"--help\"]) \n             ]]] \n             Usage: datasette publish heroku [OPTIONS] [FILES]...\n\n  Publish databases to Datasette running on Heroku\n\nOptions:\n  -m, --metadata FILENAME         Path to JSON/YAML file containing metadata to\n                                  publish\n  --extra-options TEXT            Extra options to pass to datasette serve\n  --branch TEXT                   Install datasette from a GitHub branch e.g.\n                                  main\n  --template-dir DIRECTORY        Path to directory containing custom templates\n  --plugins-dir DIRECTORY         Path to directory containing custom plugins\n  --static MOUNT:DIRECTORY        Serve static files from this directory at\n                                  /MOUNT/...\n  --install TEXT                  Additional packages (e.g. plugins) to install\n  --plugin-secret <TEXT TEXT TEXT>...\n                                  Secrets to pass to plugins, e.g. --plugin-\n                                  secret datasette-auth-github client_id xxx\n  --version-note TEXT             Additional note to show on /-/versions\n  --secret TEXT                   Secret used for signing secure values, such as\n                                  signed cookies\n  --title TEXT                    Title for metadata\n  --license TEXT                  License label for metadata\n  --license_url TEXT              License URL for metadata\n  --source TEXT                   Source label for metadata\n  --source_url TEXT               Source URL for metadata\n  --about TEXT                    About label for metadata\n  --about_url TEXT                About URL for metadata\n  -n, --name TEXT                 Application name to use when deploying\n  --tar TEXT                      --tar option to pass to Heroku, e.g.\n                                  --tar=/usr/local/bin/gtar\n  --generate-dir DIRECTORY        Output generated application files and stop\n                                  without deploying\n  --help                          Show this message and exit. \n             [[[end]]]", "breadcrumbs": "[\"CLI reference\"]", "references": "[]"}
{"id": "changelog:id57", "page": "changelog", "ref": "id57", "title": "0.44 (2020-06-11)", "content": "See also  Datasette 0.44: The annotated release notes . \n             Authentication and permissions, writable canned queries, flash messages, new plugin hooks and more.", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://simonwillison.net/2020/Jun/12/annotated-release-notes/\", \"label\": \"Datasette 0.44: The annotated release notes\"}]"}
{"id": "changelog:id55", "page": "changelog", "ref": "id55", "title": "0.45 (2020-07-01)", "content": "See also  Datasette 0.45: The annotated release notes . \n             Magic parameters for canned queries, a log out feature, improved plugin documentation and four new plugin hooks.", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://simonwillison.net/2020/Jul/1/datasette-045/\", \"label\": \"Datasette 0.45: The annotated release notes\"}]"}
{"id": "changelog:id48", "page": "changelog", "ref": "id48", "title": "0.49 (2020-09-14)", "content": "See also  Datasette 0.49: The annotated release notes . \n             \n                 \n                     Writable canned queries now expose a JSON API, see  JSON API for writable canned queries . ( #880 ) \n                 \n                 \n                     New mechanism for defining page templates with custom path parameters - a template file called  pages/about/{slug}.html  will be used to render any requests to  /about/something . See  Path parameters for pages . ( #944 ) \n                 \n                 \n                     register_output_renderer()  render functions can now return a  Response . ( #953 ) \n                 \n                 \n                     New  --upgrade  option for  datasette install . ( #945 ) \n                 \n                 \n                     New  datasette --pdb  option. ( #962 ) \n                 \n                 \n                     datasette --get  exit code now reflects the internal HTTP status code. ( #947 ) \n                 \n                 \n                     New  raise_404()  template function for returning 404 errors. ( #964 ) \n                 \n                 \n                     datasette publish heroku  now deploys using Python 3.8.5 \n                 \n                 \n                     Upgraded  CodeMirror  to 5.57.0. ( #948 ) \n                 \n                 \n                     Upgraded code style to Black 20.8b1. ( #958 ) \n                 \n                 \n                     Fixed bug where selected facets were not correctly persisted in hidden form fields on the table page. ( #963 ) \n                 \n                 \n                     Renamed the default error template from  500.html  to  error.html . \n                 \n                 \n                     Custom error pages are now documented, see  Custom error pages . ( #965 )", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://simonwillison.net/2020/Sep/15/datasette-0-49/\", \"label\": \"Datasette 0.49: The annotated release notes\"}, {\"href\": \"https://github.com/simonw/datasette/issues/880\", \"label\": \"#880\"}, {\"href\": \"https://github.com/simonw/datasette/issues/944\", \"label\": \"#944\"}, {\"href\": \"https://github.com/simonw/datasette/issues/953\", \"label\": \"#953\"}, {\"href\": \"https://github.com/simonw/datasette/issues/945\", \"label\": \"#945\"}, {\"href\": \"https://github.com/simonw/datasette/issues/962\", \"label\": \"#962\"}, {\"href\": \"https://github.com/simonw/datasette/issues/947\", \"label\": \"#947\"}, {\"href\": \"https://github.com/simonw/datasette/issues/964\", \"label\": \"#964\"}, {\"href\": \"https://codemirror.net/\", \"label\": \"CodeMirror\"}, {\"href\": \"https://github.com/simonw/datasette/issues/948\", \"label\": \"#948\"}, {\"href\": \"https://github.com/simonw/datasette/issues/958\", \"label\": \"#958\"}, {\"href\": \"https://github.com/simonw/datasette/issues/963\", \"label\": \"#963\"}, {\"href\": \"https://github.com/simonw/datasette/issues/965\", \"label\": \"#965\"}]"}
{"id": "settings:setting-cache-size-kb", "page": "settings", "ref": "setting-cache-size-kb", "title": "cache_size_kb", "content": "Sets the amount of memory SQLite uses for its  per-connection cache , in KB. \n                 datasette mydatabase.db --setting cache_size_kb 5000", "breadcrumbs": "[\"Settings\", \"Settings\"]", "references": "[{\"href\": \"https://www.sqlite.org/pragma.html#pragma_cache_size\", \"label\": \"per-connection cache\"}]"}
{"id": "binary_data:binary-plugins", "page": "binary_data", "ref": "binary-plugins", "title": "Binary plugins", "content": "Several Datasette plugins are available that change the way Datasette treats binary data. \n             \n                 \n                     datasette-render-binary  modifies Datasette's default interface to show an automatic guess at what type of binary data is being stored, along with a visual representation of the binary value that displays ASCII strings directly in the interface. \n                 \n                 \n                     datasette-render-images  detects common image formats and renders them as images directly in the Datasette interface. \n                 \n                 \n                     datasette-media  allows Datasette interfaces to be configured to serve binary files from configured SQL queries, and includes the ability to resize images directly before serving them.", "breadcrumbs": "[\"Binary data\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette-render-binary\", \"label\": \"datasette-render-binary\"}, {\"href\": \"https://github.com/simonw/datasette-render-images\", \"label\": \"datasette-render-images\"}, {\"href\": \"https://github.com/simonw/datasette-media\", \"label\": \"datasette-media\"}]"}
{"id": "settings:setting-suggest-facets", "page": "settings", "ref": "setting-suggest-facets", "title": "suggest_facets", "content": "Should Datasette calculate suggested facets? On by default, turn this off like so: \n                 datasette mydatabase.db --setting suggest_facets off", "breadcrumbs": "[\"Settings\", \"Settings\"]", "references": "[]"}
{"id": "settings:setting-allow-download", "page": "settings", "ref": "setting-allow-download", "title": "allow_download", "content": "Should users be able to download the original SQLite database using a link on the database index page? This is turned on by default. However, databases can only be downloaded if they are served in immutable mode and not in-memory. If downloading is unavailable for either of these reasons, the download link is hidden even if  allow_download  is on. To disable database downloads, use the following: \n                 datasette mydatabase.db --setting allow_download off", "breadcrumbs": "[\"Settings\", \"Settings\"]", "references": "[]"}
{"id": "settings:setting-default-allow-sql", "page": "settings", "ref": "setting-default-allow-sql", "title": "default_allow_sql", "content": "Should users be able to execute arbitrary SQL queries by default? \n                 Setting this to  off  causes permission checks for  execute-sql  to fail by default. \n                 datasette mydatabase.db --setting default_allow_sql off \n                 There are two ways to achieve this: the other is to add  \"allow_sql\": false  to your  metadata.json  file, as described in  Controlling the ability to execute arbitrary SQL . This setting offers a more convenient way to do this.", "breadcrumbs": "[\"Settings\", \"Settings\"]", "references": "[]"}
{"id": "changelog:id37", "page": "changelog", "ref": "id37", "title": "0.52.4 (2020-12-05)", "content": "Show  pysqlite3  version on  /-/versions , if installed. ( #1125 ) \n                 \n                 \n                     Errors output by Datasette (e.g. for invalid SQL queries) now go to  stderr , not  stdout . ( #1131 ) \n                 \n                 \n                     Fix for a startup error on windows caused by unnecessary  from os import EX_CANTCREAT  - thanks, Abdussamet Ko\u00e7ak.  ( #1094 )", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/coleifer/pysqlite3\", \"label\": \"pysqlite3\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1125\", \"label\": \"#1125\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1131\", \"label\": \"#1131\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1094\", \"label\": \"#1094\"}]"}
{"id": "cli-reference:cli-help-publish-help", "page": "cli-reference", "ref": "cli-help-publish-help", "title": "datasette publish", "content": "Shows a list of available deployment targets for  publishing data  with Datasette. \n             Additional deployment targets can be added by plugins that use the  publish_subcommand(publish)  hook. \n             [[[cog\nhelp([\"publish\", \"--help\"]) \n             ]]] \n             Usage: datasette publish [OPTIONS] COMMAND [ARGS]...\n\n  Publish specified SQLite database files to the internet along with a\n  Datasette-powered interface and API\n\nOptions:\n  --help  Show this message and exit.\n\nCommands:\n  cloudrun  Publish databases to Datasette running on Cloud Run\n  heroku    Publish databases to Datasette running on Heroku \n             [[[end]]]", "breadcrumbs": "[\"CLI reference\"]", "references": "[]"}
{"id": "introspection:jsondataview-plugins", "page": "introspection", "ref": "jsondataview-plugins", "title": "/-/plugins", "content": "Shows a list of currently installed plugins and their versions.  Plugins example : \n             [\n    {\n        \"name\": \"datasette_cluster_map\",\n        \"static\": true,\n        \"templates\": false,\n        \"version\": \"0.10\",\n        \"hooks\": [\"extra_css_urls\", \"extra_js_urls\", \"extra_body_script\"]\n    }\n] \n             Add  ?all=1  to include details of the default plugins baked into Datasette.", "breadcrumbs": "[\"Introspection\"]", "references": "[{\"href\": \"https://san-francisco.datasettes.com/-/plugins\", \"label\": \"Plugins example\"}]"}
{"id": "introspection:jsondataview-databases", "page": "introspection", "ref": "jsondataview-databases", "title": "/-/databases", "content": "Shows currently attached databases.  Databases example : \n             [\n    {\n        \"hash\": null,\n        \"is_memory\": false,\n        \"is_mutable\": true,\n        \"name\": \"fixtures\",\n        \"path\": \"fixtures.db\",\n        \"size\": 225280\n    }\n]", "breadcrumbs": "[\"Introspection\"]", "references": "[{\"href\": \"https://latest.datasette.io/-/databases\", \"label\": \"Databases example\"}]"}
{"id": "introspection:jsondataview-threads", "page": "introspection", "ref": "jsondataview-threads", "title": "/-/threads", "content": "Shows details of threads and  asyncio  tasks.  Threads example : \n             {\n    \"num_threads\": 2,\n    \"threads\": [\n        {\n            \"daemon\": false,\n            \"ident\": 4759197120,\n            \"name\": \"MainThread\"\n        },\n        {\n            \"daemon\": true,\n            \"ident\": 123145319682048,\n            \"name\": \"Thread-1\"\n        },\n    ],\n    \"num_tasks\": 3,\n    \"tasks\": [\n        \"<Task pending coro=<RequestResponseCycle.run_asgi() running at uvicorn/protocols/http/httptools_impl.py:385> cb=[set.discard()]>\",\n        \"<Task pending coro=<Server.serve() running at uvicorn/main.py:361> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x10365c3d0>()]> cb=[run_until_complete.<locals>.<lambda>()]>\",\n        \"<Task pending coro=<LifespanOn.main() running at uvicorn/lifespan/on.py:48> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x10364f050>()]>>\"\n    ]\n}", "breadcrumbs": "[\"Introspection\"]", "references": "[{\"href\": \"https://latest.datasette.io/-/threads\", \"label\": \"Threads example\"}]"}
{"id": "introspection:jsondataview-config", "page": "introspection", "ref": "jsondataview-config", "title": "/-/settings", "content": "Shows the  Settings  for this instance of Datasette.  Settings example : \n             {\n    \"default_facet_size\": 30,\n    \"default_page_size\": 100,\n    \"facet_suggest_time_limit_ms\": 50,\n    \"facet_time_limit_ms\": 1000,\n    \"max_returned_rows\": 1000,\n    \"sql_time_limit_ms\": 1000\n}", "breadcrumbs": "[\"Introspection\"]", "references": "[{\"href\": \"https://fivethirtyeight.datasettes.com/-/settings\", \"label\": \"Settings example\"}]"}
{"id": "introspection:jsondataview-metadata", "page": "introspection", "ref": "jsondataview-metadata", "title": "/-/metadata", "content": "Shows the contents of the  metadata.json  file that was passed to  datasette serve , if any.  Metadata example : \n             {\n    \"license\": \"CC Attribution 4.0 License\",\n    \"license_url\": \"http://creativecommons.org/licenses/by/4.0/\",\n    \"source\": \"fivethirtyeight/data on GitHub\",\n    \"source_url\": \"https://github.com/fivethirtyeight/data\",\n    \"title\": \"Five Thirty Eight\",\n    \"databases\": {\n\n    }\n}", "breadcrumbs": "[\"Introspection\"]", "references": "[{\"href\": \"https://fivethirtyeight.datasettes.com/-/metadata\", \"label\": \"Metadata example\"}]"}
{"id": "introspection:jsondataview-actor", "page": "introspection", "ref": "jsondataview-actor", "title": "/-/actor", "content": "Shows the currently authenticated actor. Useful for debugging Datasette authentication plugins. \n             {\n    \"actor\": {\n        \"id\": 1,\n        \"username\": \"some-user\"\n    }\n}", "breadcrumbs": "[\"Introspection\"]", "references": "[]"}
{"id": "introspection:jsondataview-versions", "page": "introspection", "ref": "jsondataview-versions", "title": "/-/versions", "content": "Shows the version of Datasette, Python and SQLite.  Versions example : \n             {\n    \"datasette\": {\n        \"version\": \"0.60\"\n    },\n    \"python\": {\n        \"full\": \"3.8.12 (default, Dec 21 2021, 10:45:09) \\n[GCC 10.2.1 20210110]\",\n        \"version\": \"3.8.12\"\n    },\n    \"sqlite\": {\n        \"extensions\": {\n            \"json1\": null\n        },\n        \"fts_versions\": [\n            \"FTS5\",\n            \"FTS4\",\n            \"FTS3\"\n        ],\n        \"compile_options\": [\n            \"COMPILER=gcc-6.3.0 20170516\",\n            \"ENABLE_FTS3\",\n            \"ENABLE_FTS4\",\n            \"ENABLE_FTS5\",\n            \"ENABLE_JSON1\",\n            \"ENABLE_RTREE\",\n            \"THREADSAFE=1\"\n        ],\n        \"version\": \"3.37.0\"\n    }\n}", "breadcrumbs": "[\"Introspection\"]", "references": "[{\"href\": \"https://latest.datasette.io/-/versions\", \"label\": \"Versions example\"}]"}
{"id": "changelog:csrf-protection", "page": "changelog", "ref": "csrf-protection", "title": "CSRF protection", "content": "Since writable canned queries are built using POST forms, Datasette now ships with  CSRF protection  ( #798 ). This applies automatically to any POST request, which means plugins need to include a  csrftoken  in any POST forms that they render. They can do that like so: \n                 <input type=\"hidden\" name=\"csrftoken\" value=\"{{ csrftoken() }}\">", "breadcrumbs": "[\"Changelog\", \"0.44 (2020-06-11)\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/798\", \"label\": \"#798\"}]"}
{"id": "deploying:deploying-buildpacks", "page": "deploying", "ref": "deploying-buildpacks", "title": "Deploying using buildpacks", "content": "Some hosting providers such as  Heroku ,  DigitalOcean App Platform  and  Scalingo  support the  Buildpacks standard  for deploying Python web applications. \n             Deploying Datasette on these platforms requires two files:  requirements.txt  and  Procfile . \n             The  requirements.txt  file lets the platform know which Python packages should be installed. It should contain  datasette  at a minimum, but can also list any Datasette plugins you wish to install - for example: \n             datasette\ndatasette-vega \n             The  Procfile  lets the hosting platform know how to run the command that serves web traffic. It should look like this: \n             web: datasette . -h 0.0.0.0 -p $PORT --cors \n             The  $PORT  environment variable is provided by the hosting platform.  --cors  enables CORS requests from JavaScript running on other websites to your domain - omit this if you don't want to allow CORS. You can add additional Datasette  Settings  options here too. \n             These two files should be enough to deploy Datasette on any host that supports buildpacks. Datasette will serve any SQLite files that are included in the root directory of the application. \n             If you want to build SQLite files or download them as part of the deployment process you can do so using a  bin/post_compile  file. For example, the following  bin/post_compile  will download an example database that will then be served by Datasette: \n             wget https://fivethirtyeight.datasettes.com/fivethirtyeight.db \n             simonw/buildpack-datasette-demo  is an example GitHub repository showing a Datasette configuration that can be deployed to a buildpack-supporting host.", "breadcrumbs": "[\"Deploying Datasette\"]", "references": "[{\"href\": \"https://www.heroku.com/\", \"label\": \"Heroku\"}, {\"href\": \"https://www.digitalocean.com/docs/app-platform/\", \"label\": \"DigitalOcean App Platform\"}, {\"href\": \"https://scalingo.com/\", \"label\": \"Scalingo\"}, {\"href\": \"https://buildpacks.io/\", \"label\": \"Buildpacks standard\"}, {\"href\": \"https://github.com/simonw/buildpack-datasette-demo\", \"label\": \"simonw/buildpack-datasette-demo\"}]"}
{"id": "contributing:contributing-documentation-cog", "page": "contributing", "ref": "contributing-documentation-cog", "title": "Running Cog", "content": "Some pages of documentation (in particular the  CLI reference ) are automatically updated using  Cog . \n                 To update these pages, run the following command: \n                 cog -r docs/*.rst", "breadcrumbs": "[\"Contributing\", \"Editing and building the documentation\"]", "references": "[{\"href\": \"https://github.com/nedbat/cog\", \"label\": \"Cog\"}]"}
{"id": "sql_queries:fragment", "page": "sql_queries", "ref": "fragment", "title": "fragment", "content": "Some plugins, such as  datasette-vega , can be configured by including additional data in the fragment hash of the URL - the bit that comes after a  #  symbol. \n                     You can set a default fragment hash that will be included in the link to the canned query from the database index page using the  \"fragment\"  key. \n                     This example demonstrates both  fragment  and  hide_sql : \n                     {\n    \"databases\": {\n        \"fixtures\": {\n            \"queries\": {\n                \"neighborhood_search\": {\n                    \"sql\": \"select neighborhood, facet_cities.name, state\\nfrom facetable join facet_cities on facetable.city_id = facet_cities.id\\nwhere neighborhood like '%' || :text || '%' order by neighborhood;\",\n                    \"fragment\": \"fragment-goes-here\",\n                    \"hide_sql\": true\n                }\n            }\n        }\n    }\n} \n                     See here  for a demo of this in action.", "breadcrumbs": "[\"Running SQL queries\", \"Canned queries\", \"Additional canned query options\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette-vega\", \"label\": \"datasette-vega\"}, {\"href\": \"https://latest.datasette.io/fixtures#queries\", \"label\": \"See here\"}]"}
{"id": "spatialite:installing-spatialite-on-linux", "page": "spatialite", "ref": "installing-spatialite-on-linux", "title": "Installing SpatiaLite on Linux", "content": "SpatiaLite is packaged for most Linux distributions. \n                 apt install spatialite-bin libsqlite3-mod-spatialite \n                 Depending on your distribution, you should be able to run Datasette something like this: \n                 datasette --load-extension=/usr/lib/x86_64-linux-gnu/mod_spatialite.so \n                 If you are unsure of the location of the module, try running  locate mod_spatialite  and see what comes back.", "breadcrumbs": "[\"SpatiaLite\", \"Installation\"]", "references": "[]"}
{"id": "spatialite:making-use-of-a-spatial-index", "page": "spatialite", "ref": "making-use-of-a-spatial-index", "title": "Making use of a spatial index", "content": "SpatiaLite spatial indexes are R*Trees. They allow you to run efficient bounding box queries using a sub-select, with a similar pattern to that used for  Searches using custom SQL . \n             In the above example, the resulting index will be called  idx_museums_point_geom . This takes the form of a SQLite virtual table. You can inspect its contents using the following query: \n             select * from idx_museums_point_geom limit 10; \n             Here's a live example:  timezones-api.datasette.io/timezones/idx_timezones_Geometry \n             \n                 \n                     \n                     \n                     \n                     \n                     \n                     \n                         \n                             \n                                 pkid \n                             \n                             \n                                 xmin \n                             \n                             \n                                 xmax \n                             \n                             \n                                 ymin \n                             \n                             \n                                 ymax \n                             \n                         \n                     \n                     \n                         \n                             \n                                 1 \n                             \n                             \n                                 -8.601725578308105 \n                             \n                             \n                                 -2.4930307865142822 \n                             \n                             \n                                 4.162120819091797 \n                             \n                             \n                                 10.74019718170166 \n                             \n                         \n                         \n                             \n                                 2 \n                             \n                             \n                                 -3.2607860565185547 \n                             \n                             \n                                 1.27329421043396 \n                             \n                             \n                                 4.539252281188965 \n                             \n                             \n                                 11.174856185913086 \n                             \n                         \n                         \n                             \n                                 3 \n                             \n                             \n                                 32.997581481933594 \n                             \n                             \n                                 47.98238754272461 \n                             \n                             \n                                 3.3974475860595703 \n                             \n                             \n                                 14.894054412841797 \n                             \n                         \n                         \n                             \n                                 4 \n                             \n                             \n                                 -8.66890811920166 \n                             \n                             \n                                 11.997337341308594 \n                             \n                             \n                                 18.9681453704834 \n                             \n                             \n                                 37.296207427978516 \n                             \n                         \n                         \n                             \n                                 5 \n                             \n                             \n                                 36.43336486816406 \n                             \n                             \n                                 43.300174713134766 \n                             \n                             \n                                 12.354820251464844 \n                             \n                             \n                                 18.070993423461914 \n                             \n                         \n                     \n                 \n             \n             You can now construct efficient bounding box queries that will make use of the index like this: \n             select * from museums where museums.rowid in (\n    SELECT pkid FROM idx_museums_point_geom\n    -- left-hand-edge of point > left-hand-edge of bbox (minx)\n    where xmin > :bbox_minx\n    -- right-hand-edge of point < right-hand-edge of bbox (maxx)\n    and xmax < :bbox_maxx\n    -- bottom-edge of point > bottom-edge of bbox (miny)\n    and ymin > :bbox_miny\n    -- top-edge of point < top-edge of bbox (maxy)\n    and ymax < :bbox_maxy\n); \n             Spatial indexes can be created against polygon columns as well as point columns, in which case they will represent the minimum bounding rectangle of that polygon. This is useful for accelerating  within  queries, as seen in the Timezones API example.", "breadcrumbs": "[\"SpatiaLite\"]", "references": "[{\"href\": \"https://timezones-api.datasette.io/timezones/idx_timezones_Geometry\", \"label\": \"timezones-api.datasette.io/timezones/idx_timezones_Geometry\"}]"}
{"id": "changelog:id32", "page": "changelog", "ref": "id32", "title": "0.55 (2021-02-18)", "content": "Support for cross-database SQL queries and built-in support for serving via HTTPS. \n             \n                 \n                     The new  --crossdb  command-line option causes Datasette to attach up to ten database files to the same  /_memory  database connection. This enables cross-database SQL queries, including the ability to use joins and unions to combine data from tables that exist in different database files. See  Cross-database queries  for details. ( #283 ) \n                 \n                 \n                     --ssl-keyfile  and  --ssl-certfile  options can be used to specify a TLS certificate, allowing Datasette to serve traffic over  https://  without needing to run it behind a separate proxy. ( #1221 ) \n                 \n                 \n                     The  /:memory:  page has been renamed (and redirected) to  /_memory  for consistency with the new  /_internal  database introduced in Datasette 0.54. ( #1205 ) \n                 \n                 \n                     Added plugin testing documentation on  Using pdb for errors thrown inside Datasette . ( #1207 ) \n                 \n                 \n                     The  official Datasette Docker image  now uses Python 3.7.10, applying  the latest security fix  for that Python version. ( #1235 )", "breadcrumbs": "[\"Changelog\"]", "references": "[{\"href\": \"https://github.com/simonw/datasette/issues/283\", \"label\": \"#283\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1221\", \"label\": \"#1221\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1205\", \"label\": \"#1205\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1207\", \"label\": \"#1207\"}, {\"href\": \"https://hub.docker.com/r/datasetteproject/datasette\", \"label\": \"official Datasette Docker image\"}, {\"href\": \"https://www.python.org/downloads/release/python-3710/\", \"label\": \"the latest security fix\"}, {\"href\": \"https://github.com/simonw/datasette/issues/1235\", \"label\": \"#1235\"}]"}
{"id": "full_text_search:full-text-search-table-view-api", "page": "full_text_search", "ref": "full-text-search-table-view-api", "title": "The table page and table view API", "content": "Table views that support full-text search can be queried using the  ?_search=TERMS  query string parameter. This will run the search against content from all of the columns that have been included in the index. \n             Try this example:  fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort \n             SQLite full-text search supports wildcards. This means you can easily implement prefix auto-complete by including an asterisk at the end of the search term - for example: \n             /dbname/tablename/?_search=rob* \n             This will return all records containing at least one word that starts with the letters  rob . \n             You can also run searches against just the content of a specific named column by using  _search_COLNAME=TERMS  - for example, this would search for just rows where the  name  column in the FTS index mentions  Sarah : \n             /dbname/tablename/?_search_name=Sarah", "breadcrumbs": "[\"Full-text search\"]", "references": "[{\"href\": \"https://fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort\", \"label\": \"fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort\"}]"}
{"id": "cli-reference:cli-datasette-get", "page": "cli-reference", "ref": "cli-datasette-get", "title": "datasette --get", "content": "The  --get  option to  datasette serve  (or just  datasette ) specifies the path to a page within Datasette and causes Datasette to output the content from that path without starting the web server. \n                 This means that all of Datasette's functionality can be accessed directly from the command-line. \n                 For example: \n                 $ datasette --get '/-/versions.json' | jq .\n{\n  \"python\": {\n    \"version\": \"3.8.5\",\n    \"full\": \"3.8.5 (default, Jul 21 2020, 10:48:26) \\n[Clang 11.0.3 (clang-1103.0.32.62)]\"\n  },\n  \"datasette\": {\n    \"version\": \"0.46+15.g222a84a.dirty\"\n  },\n  \"asgi\": \"3.0\",\n  \"uvicorn\": \"0.11.8\",\n  \"sqlite\": {\n    \"version\": \"3.32.3\",\n    \"fts_versions\": [\n      \"FTS5\",\n      \"FTS4\",\n      \"FTS3\"\n    ],\n    \"extensions\": {\n      \"json1\": null\n    },\n    \"compile_options\": [\n      \"COMPILER=clang-11.0.3\",\n      \"ENABLE_COLUMN_METADATA\",\n      \"ENABLE_FTS3\",\n      \"ENABLE_FTS3_PARENTHESIS\",\n      \"ENABLE_FTS4\",\n      \"ENABLE_FTS5\",\n      \"ENABLE_GEOPOLY\",\n      \"ENABLE_JSON1\",\n      \"ENABLE_PREUPDATE_HOOK\",\n      \"ENABLE_RTREE\",\n      \"ENABLE_SESSION\",\n      \"MAX_VARIABLE_NUMBER=250000\",\n      \"THREADSAFE=1\"\n    ]\n  }\n} \n                 The exit code will be 0 if the request succeeds and 1 if the request produced an HTTP status code other than 200 - e.g. a 404 or 500 error. \n                 This lets you use  datasette --get /  to run tests against a Datasette application in a continuous integration environment such as GitHub Actions.", "breadcrumbs": "[\"CLI reference\", \"datasette serve\"]", "references": "[]"}
{"id": "binary_data:binary-linking", "page": "binary_data", "ref": "binary-linking", "title": "Linking to binary downloads", "content": "The  .blob  output format is used to return binary data. It requires a  _blob_column=  query string argument specifying which BLOB column should be downloaded, for example: \n             https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data \n             This output format can also be used to return binary data from an arbitrary SQL query. Since such queries do not specify an exact row, an additional  ?_blob_hash=  parameter can be used to specify the SHA-256 hash of the value that is being linked to. \n             Consider the query  select data from binary_data  -  demonstrated here . \n             That page links to the binary value downloads. Those links look like this: \n             https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d \n             These  .blob  links are also returned in the  .csv  exports Datasette provides for binary tables and queries, since the CSV format does not have a mechanism for representing binary data.", "breadcrumbs": "[\"Binary data\"]", "references": "[{\"href\": \"https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data\", \"label\": \"https://latest.datasette.io/fixtures/binary_data/1.blob?_blob_column=data\"}, {\"href\": \"https://latest.datasette.io/fixtures?sql=select+data+from+binary_data\", \"label\": \"demonstrated here\"}, {\"href\": \"https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d\", \"label\": \"https://latest.datasette.io/fixtures.blob?sql=select+data+from+binary_data&_blob_column=data&_blob_hash=f3088978da8f9aea479ffc7f631370b968d2e855eeb172bea7f6c7a04262bb6d\"}]"}
{"id": "authentication:allowdebugview", "page": "authentication", "ref": "allowdebugview", "title": "The /-/allow-debug tool", "content": "The  /-/allow-debug  tool lets you try out different   \"action\"  blocks against different  \"actor\"  JSON objects. You can try that out here:  https://latest.datasette.io/-/allow-debug", "breadcrumbs": "[\"Authentication and permissions\", \"Permissions\"]", "references": "[{\"href\": \"https://latest.datasette.io/-/allow-debug\", \"label\": \"https://latest.datasette.io/-/allow-debug\"}]"}