id,page,ref,title,content,breadcrumbs,references custom_templates:custom-pages-errors,custom_templates,custom-pages-errors,Custom error pages,"Datasette returns an error page if an unexpected error occurs, access is forbidden or content cannot be found. You can customize the response returned for these errors by providing a custom error page template. Content not found errors use a 404.html template. Access denied errors use 403.html . Invalid input errors use 400.html . Unexpected errors of other kinds use 500.html . If a template for the specific error code is not found a template called error.html will be used instead. If you do not provide that template Datasette's default error.html template will be used. The error template will be passed the following context: status - integer The integer HTTP status code, e.g. 404, 500, 403, 400. error - string Details of the specific error, usually a full sentence. title - string or None A title for the page representing the class of error. This is often None for errors that do not provide a title separate from their error message.","[""Custom pages and templates""]","[{""href"": ""https://github.com/simonw/datasette/blob/main/datasette/templates/error.html"", ""label"": ""default error.html template""}]" performance:performance,performance,performance,Performance and caching,"Datasette runs on top of SQLite, and SQLite has excellent performance. For small databases almost any query should return in just a few milliseconds, and larger databases (100s of MBs or even GBs of data) should perform extremely well provided your queries make sensible use of database indexes. That said, there are a number of tricks you can use to improve Datasette's performance.",[],[] ecosystem:ecosystem,ecosystem,ecosystem,The Datasette Ecosystem,"Datasette sits at the center of a growing ecosystem of open source tools aimed at making it as easy as possible to gather, analyze and publish interesting data. These tools are divided into two main groups: tools for building SQLite databases (for use with Datasette) and plugins that extend Datasette's functionality. The Datasette project website includes a directory of plugins and a directory of tools: Plugins directory on datasette.io Tools directory on datasette.io",[],"[{""href"": ""https://datasette.io/"", ""label"": ""Datasette project website""}, {""href"": ""https://datasette.io/plugins"", ""label"": ""Plugins directory on datasette.io""}, {""href"": ""https://datasette.io/tools"", ""label"": ""Tools directory on datasette.io""}]" settings:using-setting,settings,using-setting,Using --setting,"Datasette supports a number of settings. These can be set using the --setting name value option to datasette serve . You can set multiple settings at once like this: datasette mydatabase.db \ --setting default_page_size 50 \ --setting sql_time_limit_ms 3500 \ --setting max_returned_rows 2000","[""Settings""]",[] metadata:specifying-units-for-a-column,metadata,specifying-units-for-a-column,Specifying units for a column,"Datasette supports attaching units to a column, which will be used when displaying values from that column. SI prefixes will be used where appropriate. Column units are configured in the metadata like so: { ""databases"": { ""database1"": { ""tables"": { ""example_table"": { ""units"": { ""column1"": ""metres"", ""column2"": ""Hz"" } } } } } } Units are interpreted using Pint , and you can see the full list of available units in Pint's unit registry . You can also add custom units to the metadata, which will be registered with Pint: { ""custom_units"": [ ""decibel = [] = dB"" ] }","[""Metadata""]","[{""href"": ""https://pint.readthedocs.io/"", ""label"": ""Pint""}, {""href"": ""https://github.com/hgrecco/pint/blob/master/pint/default_en.txt"", ""label"": ""unit registry""}, {""href"": ""http://pint.readthedocs.io/en/latest/defining.html"", ""label"": ""custom units""}]" full_text_search:full-text-search-enabling,full_text_search,full-text-search-enabling,Enabling full-text search for a SQLite table,"Datasette takes advantage of the external content mechanism in SQLite, which allows a full-text search virtual table to be associated with the contents of another SQLite table. To set up full-text search for a table, you need to do two things: Create a new FTS virtual table associated with your table Populate that FTS table with the data that you would like to be able to run searches against","[""Full-text search""]","[{""href"": ""https://www.sqlite.org/fts3.html#_external_content_fts4_tables_"", ""label"": ""external content""}]" sql_queries:sql,sql_queries,sql,Running SQL queries,"Datasette treats SQLite database files as read-only and immutable. This means it is not possible to execute INSERT or UPDATE statements using Datasette, which allows us to expose SELECT statements to the outside world without needing to worry about SQL injection attacks. The easiest way to execute custom SQL against Datasette is through the web UI. The database index page includes a SQL editor that lets you run any SELECT query you like. You can also construct queries using the filter interface on the tables page, then click ""View and edit SQL"" to open that query in the custom SQL editor. Note that this interface is only available if the execute-sql permission is allowed. Any Datasette SQL query is reflected in the URL of the page, allowing you to bookmark them, share them with others and navigate through previous queries using your browser back button. You can also retrieve the results of any query as JSON by adding .json to the base URL.",[],[] internals:internals-csrf,internals,internals-csrf,CSRF protection,"Datasette uses asgi-csrf to guard against CSRF attacks on form POST submissions. Users receive a ds_csrftoken cookie which is compared against the csrftoken form field (or x-csrftoken HTTP header) for every incoming request. If your plugin implements a
anywhere you will need to include that token. You can do so with the following template snippet: If you are rendering templates using the await .render_template(template, context=None, request=None) method the csrftoken() helper will only work if you provide the request= argument to that method. If you forget to do this you will see the following error: form-urlencoded POST field did not match cookie You can selectively disable CSRF protection using the skip_csrf(datasette, scope) hook.","[""Internals for plugins""]","[{""href"": ""https://github.com/simonw/asgi-csrf"", ""label"": ""asgi-csrf""}]" internals:internals-tilde-encoding,internals,internals-tilde-encoding,Tilde encoding,"Datasette uses a custom encoding scheme in some places, called tilde encoding . This is primarily used for table names and row primary keys, to avoid any confusion between / characters in those values and the Datasette URLs that reference them. Tilde encoding uses the same algorithm as URL percent-encoding , but with the ~ tilde character used in place of % . Any character other than ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz0123456789_- will be replaced by the numeric equivalent preceded by a tilde. For example: / becomes ~2F . becomes ~2E % becomes ~25 ~ becomes ~7E Space becomes + polls/2022.primary becomes polls~2F2022~2Eprimary Note that the space character is a special case: it will be replaced with a + symbol. datasette.utils. tilde_encode s : str str Returns tilde-encoded string - for example /foo/bar -> ~2Ffoo~2Fbar datasette.utils. tilde_decode s : str str Decodes a tilde-encoded string, so ~2Ffoo~2Fbar -> /foo/bar","[""Internals for plugins"", ""The datasette.utils module""]","[{""href"": ""https://developer.mozilla.org/en-US/docs/Glossary/percent-encoding"", ""label"": ""URL percent-encoding""}]" settings:setting-secret,settings,setting-secret,Configuring the secret,"Datasette uses a secret string to sign secure values such as cookies. If you do not provide a secret, Datasette will create one when it starts up. This secret will reset every time the Datasette server restarts though, so things like authentication cookies will not stay valid between restarts. You can pass a secret to Datasette in two ways: with the --secret command-line option or by setting a DATASETTE_SECRET environment variable. $ datasette mydb.db --secret=SECRET_VALUE_HERE Or: $ export DATASETTE_SECRET=SECRET_VALUE_HERE $ datasette mydb.db One way to generate a secure random secret is to use Python like this: $ python3 -c 'import secrets; print(secrets.token_hex(32))' cdb19e94283a20f9d42cca50c5a4871c0aa07392db308755d60a1a5b9bb0fa52 Plugin authors make use of this signing mechanism in their plugins using .sign(value, namespace=""default"") and .unsign(value, namespace=""default"") .","[""Settings""]",[] contributing:contributing-formatting,contributing,contributing-formatting,Code formatting,"Datasette uses opinionated code formatters: Black for Python and Prettier for JavaScript. These formatters are enforced by Datasette's continuous integration: if a commit includes Python or JavaScript code that does not match the style enforced by those tools, the tests will fail. When developing locally, you can verify and correct the formatting of your code using these tools.","[""Contributing""]","[{""href"": ""https://github.com/psf/black"", ""label"": ""Black""}, {""href"": ""https://prettier.io/"", ""label"": ""Prettier""}]" changelog:writable-canned-queries,changelog,writable-canned-queries,Writable canned queries,"Datasette's Canned queries feature lets you define SQL queries in metadata.json which can then be executed by users visiting a specific URL. https://latest.datasette.io/fixtures/neighborhood_search for example. Canned queries were previously restricted to SELECT , but Datasette 0.44 introduces the ability for canned queries to execute INSERT or UPDATE queries as well, using the new ""write"": true property ( #800 ): { ""databases"": { ""dogs"": { ""queries"": { ""add_name"": { ""sql"": ""INSERT INTO names (name) VALUES (:name)"", ""write"": true } } } } } See Writable canned queries for more details.","[""Changelog"", ""0.44 (2020-06-11)""]","[{""href"": ""https://latest.datasette.io/fixtures/neighborhood_search"", ""label"": ""https://latest.datasette.io/fixtures/neighborhood_search""}, {""href"": ""https://github.com/simonw/datasette/issues/800"", ""label"": ""#800""}]" changelog:new-configuration-settings,changelog,new-configuration-settings,New configuration settings,"Datasette's Settings now also supports boolean settings. A number of new configuration options have been added: num_sql_threads - the number of threads used to execute SQLite queries. Defaults to 3. allow_facet - enable or disable custom Facets using the _facet= parameter. Defaults to on. suggest_facets - should Datasette suggest facets? Defaults to on. allow_download - should users be allowed to download the entire SQLite database? Defaults to on. allow_sql - should users be allowed to execute custom SQL queries? Defaults to on. default_cache_ttl - Default HTTP caching max-age header in seconds. Defaults to 365 days - caching can be disabled entirely by settings this to 0. cache_size_kb - Set the amount of memory SQLite uses for its per-connection cache , in KB. allow_csv_stream - allow users to stream entire result sets as a single CSV file. Defaults to on. max_csv_mb - maximum size of a returned CSV file in MB. Defaults to 100MB, set to 0 to disable this limit.","[""Changelog"", ""0.23 (2018-06-18)""]","[{""href"": ""https://www.sqlite.org/pragma.html#pragma_cache_size"", ""label"": ""per-connection cache""}]" metadata:label-columns,metadata,label-columns,Specifying the label column for a table,"Datasette's HTML interface attempts to display foreign key references as labelled hyperlinks. By default, it looks for referenced tables that only have two columns: a primary key column and one other. It assumes that the second column should be used as the link label. If your table has more than two columns you can specify which column should be used for the link label with the label_column property: { ""databases"": { ""database1"": { ""tables"": { ""example_table"": { ""label_column"": ""title"" } } } } }","[""Metadata""]",[] sql_queries:id2,sql_queries,id2,Pagination,"Datasette's default table pagination is designed to be extremely efficient. SQL OFFSET/LIMIT pagination can have a significant performance penalty once you get into multiple thousands of rows, as each page still requires the database to scan through every preceding row to find the correct offset. When paginating through tables, Datasette instead orders the rows in the table by their primary key and performs a WHERE clause against the last seen primary key for the previous page. For example: select rowid, * from Tree_List where rowid > 200 order by rowid limit 101 This represents page three for this particular table, with a page size of 100. Note that we request 101 items in the limit clause rather than 100. This allows us to detect if we are on the last page of the results: if the query returns less than 101 rows we know we have reached the end of the pagination set. Datasette will only return the first 100 rows - the 101st is used purely to detect if there should be another page. Since the where clause acts against the index on the primary key, the query is extremely fast even for records that are a long way into the overall pagination set.","[""Running SQL queries""]",[] contributing:contributing-documentation,contributing,contributing-documentation,Editing and building the documentation,"Datasette's documentation lives in the docs/ directory and is deployed automatically using Read The Docs . The documentation is written using reStructuredText. You may find this article on The subset of reStructuredText worth committing to memory useful. You can build it locally by installing sphinx and sphinx_rtd_theme in your Datasette development environment and then running make html directly in the docs/ directory: # You may first need to activate your virtual environment: source venv/bin/activate # Install the dependencies needed to build the docs pip install -e .[docs] # Now build the docs cd docs/ make html This will create the HTML version of the documentation in docs/_build/html . You can open it in your browser like so: open _build/html/index.html Any time you make changes to a .rst file you can re-run make html to update the built documents, then refresh them in your browser. For added productivity, you can use use sphinx-autobuild to run Sphinx in auto-build mode. This will run a local webserver serving the docs that automatically rebuilds them and refreshes the page any time you hit save in your editor. sphinx-autobuild will have been installed when you ran pip install -e .[docs] . In your docs/ directory you can start the server by running the following: make livehtml Now browse to http://localhost:8000/ to view the documentation. Any edits you make should be instantly reflected in your browser.","[""Contributing""]","[{""href"": ""https://readthedocs.org/"", ""label"": ""Read The Docs""}, {""href"": ""https://simonwillison.net/2018/Aug/25/restructuredtext/"", ""label"": ""The subset of reStructuredText worth committing to memory""}, {""href"": ""https://pypi.org/project/sphinx-autobuild/"", ""label"": ""sphinx-autobuild""}]" plugins:id1,plugins,id1,Plugins,"Datasette's plugin system allows additional features to be implemented as Python code (or front-end JavaScript) which can be wrapped up in a separate Python package. The underlying mechanism uses pluggy . See the Datasette plugins directory for a list of existing plugins, or take a look at the datasette-plugin topic on GitHub. Things you can do with plugins include: Add visualizations to Datasette, for example datasette-cluster-map and datasette-vega . Make new custom SQL functions available for use within Datasette, for example datasette-haversine and datasette-jellyfish . Define custom output formats with custom extensions, for example datasette-atom and datasette-ics . Add template functions that can be called within your Jinja custom templates, for example datasette-render-markdown . Customize how database values are rendered in the Datasette interface, for example datasette-render-binary and datasette-pretty-json . Customize how Datasette's authentication and permissions systems work, for example datasette-auth-tokens and datasette-permissions-sql .",[],"[{""href"": ""https://pluggy.readthedocs.io/"", ""label"": ""pluggy""}, {""href"": ""https://datasette.io/plugins"", ""label"": ""Datasette plugins directory""}, {""href"": ""https://github.com/topics/datasette-plugin"", ""label"": ""datasette-plugin""}, {""href"": ""https://github.com/simonw/datasette-cluster-map"", ""label"": ""datasette-cluster-map""}, {""href"": ""https://github.com/simonw/datasette-vega"", ""label"": ""datasette-vega""}, {""href"": ""https://github.com/simonw/datasette-haversine"", ""label"": ""datasette-haversine""}, {""href"": ""https://github.com/simonw/datasette-jellyfish"", ""label"": ""datasette-jellyfish""}, {""href"": ""https://github.com/simonw/datasette-atom"", ""label"": ""datasette-atom""}, {""href"": ""https://github.com/simonw/datasette-ics"", ""label"": ""datasette-ics""}, {""href"": ""https://github.com/simonw/datasette-render-markdown#markdown-in-templates"", ""label"": ""datasette-render-markdown""}, {""href"": ""https://github.com/simonw/datasette-render-binary"", ""label"": ""datasette-render-binary""}, {""href"": ""https://github.com/simonw/datasette-pretty-json"", ""label"": ""datasette-pretty-json""}, {""href"": ""https://github.com/simonw/datasette-auth-tokens"", ""label"": ""datasette-auth-tokens""}, {""href"": ""https://github.com/simonw/datasette-permissions-sql"", ""label"": ""datasette-permissions-sql""}]" facets:suggested-facets,facets,suggested-facets,Suggested facets,"Datasette's table UI will suggest facets for the user to apply, based on the following criteria: For the currently filtered data are there any columns which, if applied as a facet... Will return 30 or less unique options Will return more than one unique option Will return less unique options than the total number of filtered rows And the query used to evaluate this criteria can be completed in under 50ms That last point is particularly important: Datasette runs a query for every column that is displayed on a page, which could get expensive - so to avoid slow load times it sets a time limit of just 50ms for each of those queries. This means suggested facets are unlikely to appear for tables with millions of records in them.","[""Facets""]",[] settings:setting-default-cache-ttl,settings,setting-default-cache-ttl,default_cache_ttl,"Default HTTP caching max-age header in seconds, used for Cache-Control: max-age=X . Can be over-ridden on a per-request basis using the ?_ttl= query string parameter. Set this to 0 to disable HTTP caching entirely. Defaults to 5 seconds. datasette mydatabase.db --setting default_cache_ttl 60","[""Settings"", ""Settings""]",[] changelog:id73,changelog,id73,0.31.1 (2019-11-12),Deployments created using datasette publish now use python:3.8 base Docker image ( #629 ),"[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/pull/629"", ""label"": ""#629""}]" changelog:id89,changelog,id89,0.25.1 (2018-11-04),"Documentation improvements plus a fix for publishing to Zeit Now. datasette publish now now uses Zeit's v1 platform, to work around the new 100MB image limit. Thanks, @slygent - closes #366 .","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/366"", ""label"": ""#366""}]" changelog:id31,changelog,id31,0.56 (2021-03-28),"Documentation improvements, bug fixes and support for SpatiaLite 5. The SQL editor can now be resized by dragging a handle. ( #1236 ) Fixed a bug with JSON faceting and the __arraycontains filter caused by tables with spaces in their names. ( #1239 ) Upgraded httpx dependency. ( #1005 ) JSON faceting is now suggested even if a column contains blank strings. ( #1246 ) New datasette.add_memory_database() method. ( #1247 ) The Response.asgi_send() method is now documented. ( #1266 ) The official Datasette Docker image now bundles SpatiaLite version 5. ( #1278 ) Fixed a no such table: pragma_database_list bug when running Datasette against SQLite versions prior to SQLite 3.16.0. ( #1276 ) HTML lists displayed in table cells are now styled correctly. Thanks, Bob Whitelock. ( #1141 , #1252 ) Configuration directory mode now correctly serves immutable databases that are listed in inspect-data.json . Thanks Campbell Allen and Frankie Robertson. ( #1031 , #1229 )","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1236"", ""label"": ""#1236""}, {""href"": ""https://github.com/simonw/datasette/issues/1239"", ""label"": ""#1239""}, {""href"": ""https://github.com/simonw/datasette/issues/1005"", ""label"": ""#1005""}, {""href"": ""https://github.com/simonw/datasette/issues/1246"", ""label"": ""#1246""}, {""href"": ""https://github.com/simonw/datasette/issues/1247"", ""label"": ""#1247""}, {""href"": ""https://github.com/simonw/datasette/issues/1266"", ""label"": ""#1266""}, {""href"": ""https://github.com/simonw/datasette/issues/1278"", ""label"": ""#1278""}, {""href"": ""https://github.com/simonw/datasette/issues/1276"", ""label"": ""#1276""}, {""href"": ""https://github.com/simonw/datasette/issues/1141"", ""label"": ""#1141""}, {""href"": ""https://github.com/simonw/datasette/pull/1252"", ""label"": ""#1252""}, {""href"": ""https://github.com/simonw/datasette/pull/1031"", ""label"": ""#1031""}, {""href"": ""https://github.com/simonw/datasette/pull/1229"", ""label"": ""#1229""}]" changelog:id7,changelog,id7,0.64.1 (2023-01-11),"Documentation now links to a current source of information for installing Python 3. ( #1987 ) Incorrectly calling the Datasette constructor using Datasette(""path/to/data.db"") instead of Datasette([""path/to/data.db""]) now returns a useful error message. ( #1985 )","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1987"", ""label"": ""#1987""}, {""href"": ""https://github.com/simonw/datasette/issues/1985"", ""label"": ""#1985""}]" changelog:id40,changelog,id40,0.52.1 (2020-11-29),"Documentation on Testing plugins now recommends using datasette.client . ( #1102 ) Fix bug where compound foreign keys produced broken links. ( #1098 ) datasette --load-module=spatialite now also checks for /usr/local/lib/mod_spatialite.so . Thanks, Dan Peterson. ( #1114 )","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1102"", ""label"": ""#1102""}, {""href"": ""https://github.com/simonw/datasette/issues/1098"", ""label"": ""#1098""}, {""href"": ""https://github.com/simonw/datasette/issues/1114"", ""label"": ""#1114""}]" ecosystem:dogsheep,ecosystem,dogsheep,Dogsheep,Dogsheep is a collection of tools for personal analytics using SQLite and Datasette. The project provides tools like github-to-sqlite and twitter-to-sqlite that can import data from different sources in order to create a personal data warehouse. Personal Data Warehouses: Reclaiming Your Data is a talk that explains Dogsheep and demonstrates it in action.,"[""The Datasette Ecosystem""]","[{""href"": ""https://dogsheep.github.io/"", ""label"": ""Dogsheep""}, {""href"": ""https://datasette.io/tools/github-to-sqlite"", ""label"": ""github-to-sqlite""}, {""href"": ""https://datasette.io/tools/twitter-to-sqlite"", ""label"": ""twitter-to-sqlite""}, {""href"": ""https://simonwillison.net/2020/Nov/14/personal-data-warehouses/"", ""label"": ""Personal Data Warehouses: Reclaiming Your Data""}]" changelog:id65,changelog,id65,0.37.1 (2020-03-02),"Don't attempt to count table rows to display on the index page for databases > 100MB. ( #688 ) Print exceptions if they occur in the write thread rather than silently swallowing them. Handle the possibility of scope[""path""] being a string rather than bytes Better documentation for the extra_template_vars(template, database, table, columns, view_name, request, datasette) plugin hook.","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/688"", ""label"": ""#688""}]" changelog:bug-fixes,changelog,bug-fixes,Bug fixes,"Don't show the facet option in the cog menu if faceting is not allowed. ( #1683 ) ?_sort and ?_sort_desc now work if the column that is being sorted has been excluded from the query using ?_col= or ?_nocol= . ( #1773 ) Fixed bug where ?_sort_desc was duplicated in the URL every time the Apply button was clicked. ( #1738 )","[""Changelog"", ""0.62 (2022-08-14)""]","[{""href"": ""https://github.com/simonw/datasette/issues/1683"", ""label"": ""#1683""}, {""href"": ""https://github.com/simonw/datasette/issues/1773"", ""label"": ""#1773""}, {""href"": ""https://github.com/simonw/datasette/issues/1738"", ""label"": ""#1738""}]" changelog:id3,changelog,id3,0.64.5 (2023-10-08),"Dropped dependency on click-default-group-wheel , which could cause a dependency conflict. ( #2197 )","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/2197"", ""label"": ""#2197""}]" pages:databaseview,pages,databaseview,Database,"Each database has a page listing the tables, views and canned queries available for that database. If the execute-sql permission is enabled (it's on by default) there will also be an interface for executing arbitrary SQL select queries against the data. Examples: fivethirtyeight.datasettes.com/fivethirtyeight global-power-plants.datasettes.com/global-power-plants The JSON version of this page provides programmatic access to the underlying data: fivethirtyeight.datasettes.com/fivethirtyeight.json global-power-plants.datasettes.com/global-power-plants.json","[""Pages and API endpoints""]","[{""href"": ""https://fivethirtyeight.datasettes.com/fivethirtyeight"", ""label"": ""fivethirtyeight.datasettes.com/fivethirtyeight""}, {""href"": ""https://global-power-plants.datasettes.com/global-power-plants"", ""label"": ""global-power-plants.datasettes.com/global-power-plants""}, {""href"": ""https://fivethirtyeight.datasettes.com/fivethirtyeight.json"", ""label"": ""fivethirtyeight.datasettes.com/fivethirtyeight.json""}, {""href"": ""https://global-power-plants.datasettes.com/global-power-plants.json"", ""label"": ""global-power-plants.datasettes.com/global-power-plants.json""}]" settings:setting-allow-csv-stream,settings,setting-allow-csv-stream,allow_csv_stream,"Enables the CSV export feature where an entire table (potentially hundreds of thousands of rows) can be exported as a single CSV file. This is turned on by default - you can turn it off like this: datasette mydatabase.db --setting allow_csv_stream off","[""Settings"", ""Settings""]",[] json_api:json-api-special,json_api,json-api-special,Special JSON arguments,"Every Datasette endpoint that can return JSON also accepts the following query string arguments: ?_shape=SHAPE The shape of the JSON to return, documented above. ?_nl=on When used with ?_shape=array produces newline-delimited JSON objects. ?_json=COLUMN1&_json=COLUMN2 If any of your SQLite columns contain JSON values, you can use one or more _json= parameters to request that those columns be returned as regular JSON. Without this argument those columns will be returned as JSON objects that have been double-encoded into a JSON string value. Compare this query without the argument to this query using the argument ?_json_infinity=on If your data contains infinity or -infinity values, Datasette will replace them with None when returning them as JSON. If you pass _json_infinity=1 Datasette will instead return them as Infinity or -Infinity which is invalid JSON but can be processed by some custom JSON parsers. ?_timelimit=MS Sets a custom time limit for the query in ms. You can use this for optimistic queries where you would like Datasette to give up if the query takes too long, for example if you want to implement autocomplete search but only if it can be executed in less than 10ms. ?_ttl=SECONDS For how many seconds should this response be cached by HTTP proxies? Use ?_ttl=0 to disable HTTP caching entirely for this request. ?_trace=1 Turns on tracing for this page: SQL queries executed during the request will be gathered and included in the response, either in a new ""_traces"" key for JSON responses or at the bottom of the page if the response is in HTML. The structure of the data returned here should be considered highly unstable and very likely to change. Only available if the trace_debug setting is enabled.","[""JSON API""]","[{""href"": ""https://fivethirtyeight.datasettes.com/fivethirtyeight.json?sql=select+%27{%22this+is%22%3A+%22a+json+object%22}%27+as+d&_shape=array"", ""label"": ""this query without the argument""}, {""href"": ""https://fivethirtyeight.datasettes.com/fivethirtyeight.json?sql=select+%27{%22this+is%22%3A+%22a+json+object%22}%27+as+d&_shape=array&_json=d"", ""label"": ""this query using the argument""}]" changelog:latest-datasette-io,changelog,latest-datasette-io,latest.datasette.io,"Every commit to Datasette master is now automatically deployed by Travis CI to https://latest.datasette.io/ - ensuring there is always a live demo of the latest version of the software. The demo uses the fixtures from our unit tests, ensuring it demonstrates the same range of functionality that is covered by the tests. You can see how the deployment mechanism works in our .travis.yml file.","[""Changelog"", ""0.23 (2018-06-18)""]","[{""href"": ""https://latest.datasette.io/"", ""label"": ""https://latest.datasette.io/""}, {""href"": ""https://github.com/simonw/datasette/blob/master/tests/fixtures.py"", ""label"": ""the fixtures""}, {""href"": ""https://github.com/simonw/datasette/blob/master/.travis.yml"", ""label"": "".travis.yml""}]" custom_templates:css-classes-on-the-body,custom_templates,css-classes-on-the-body,CSS classes on the ,"Every default template includes CSS classes in the body designed to support custom styling. The index template (the top level page at / ) gets this: The database template ( /dbname ) gets this: The custom SQL template ( /dbname?sql=... ) gets this: A canned query template ( /dbname/queryname ) gets this: The table template ( /dbname/tablename ) gets: The row template ( /dbname/tablename/rowid ) gets: The db-x and table-x classes use the database or table names themselves if they are valid CSS identifiers. If they aren't, we strip any invalid characters out and append a 6 character md5 digest of the original name, in order to ensure that multiple tables which resolve to the same stripped character version still have different CSS classes. Some examples: ""simple"" => ""simple"" ""MixedCase"" => ""MixedCase"" ""-no-leading-hyphens"" => ""no-leading-hyphens-65bea6"" ""_no-leading-underscores"" => ""no-leading-underscores-b921bc"" ""no spaces"" => ""no-spaces-7088d7"" ""-"" => ""336d5e"" ""no $ characters"" => ""no--characters-59e024"" and elements also get custom CSS classes reflecting the database column they are representing, for example:
id name
1 SMITH
","[""Custom pages and templates"", ""Custom CSS and JavaScript""]",[] pages:rowview,pages,rowview,Row,"Every row in every Datasette table has its own URL. This means individual records can be linked to directly. Table cells with extremely long text contents are truncated on the table view according to the truncate_cells_html setting. If a cell has been truncated the full length version of that cell will be available on the row page. Rows which are the targets of foreign key references from other tables will show a link to a filtered search for all records that reference that row. Here's an example from the Registers of Members Interests database: ../people/uk.org.publicwhip%2Fperson%2F10001 Note that this URL includes the encoded primary key of the record. Here's that same page as JSON: ../people/uk.org.publicwhip%2Fperson%2F10001.json","[""Pages and API endpoints""]","[{""href"": ""https://register-of-members-interests.datasettes.com/regmem/people/uk.org.publicwhip%2Fperson%2F10001"", ""label"": ""../people/uk.org.publicwhip%2Fperson%2F10001""}, {""href"": ""https://register-of-members-interests.datasettes.com/regmem/people/uk.org.publicwhip%2Fperson%2F10001.json"", ""label"": ""../people/uk.org.publicwhip%2Fperson%2F10001.json""}]" changelog:id15,changelog,id15,Documentation,"Examples in the documentation now include a copy-to-clipboard button. ( #1748 ) Documentation now uses the Furo Sphinx theme. ( #1746 ) Code examples in the documentation are now all formatted using Black. ( #1718 ) Request.fake() method is now documented, see Request object . New documentation for plugin authors: Registering a plugin for the duration of a test . ( #903 )","[""Changelog"", ""0.62 (2022-08-14)""]","[{""href"": ""https://github.com/simonw/datasette/issues/1748"", ""label"": ""#1748""}, {""href"": ""https://github.com/pradyunsg/furo"", ""label"": ""Furo""}, {""href"": ""https://github.com/simonw/datasette/issues/1746"", ""label"": ""#1746""}, {""href"": ""https://github.com/simonw/datasette/issues/1718"", ""label"": ""#1718""}, {""href"": ""https://github.com/simonw/datasette/issues/903"", ""label"": ""#903""}]" internals:database-execute,internals,database-execute,"await db.execute(sql, ...)","Executes a SQL query against the database and returns the resulting rows (see Results ). sql - string (required) The SQL query to execute. This can include ? or :named parameters. params - list or dict A list or dictionary of values to use for the parameters. List for ? , dictionary for :named . truncate - boolean Should the rows returned by the query be truncated at the maximum page size? Defaults to True , set this to False to disable truncation. custom_time_limit - integer ms A custom time limit for this query. This can be set to a lower value than the Datasette configured default. If a query takes longer than this it will be terminated early and raise a dataette.database.QueryInterrupted exception. page_size - integer Set a custom page size for truncation, over-riding the configured Datasette default. log_sql_errors - boolean Should any SQL errors be logged to the console in addition to being raised as an error? Defaults to True .","[""Internals for plugins"", ""Database class""]",[] internals:database-execute-fn,internals,database-execute-fn,await db.execute_fn(fn),"Executes a given callback function against a read-only database connection running in a thread. The function will be passed a SQLite connection, and the return value from the function will be returned by the await . Example usage: def get_version(conn): return conn.execute( ""select sqlite_version()"" ).fetchall()[0][0] version = await db.execute_fn(get_version)","[""Internals for plugins"", ""Database class""]",[] plugin_hooks:plugin-hook-extra-body-script,plugin_hooks,plugin-hook-extra-body-script,"extra_body_script(template, database, table, columns, view_name, request, datasette)","Extra JavaScript to be added to a element: @hookimpl def extra_body_script(): return { ""module"": True, ""script"": ""console.log('Your JavaScript goes here...')"", } This will add the following to the end of your page: Example: datasette-cluster-map","[""Plugin hooks""]","[{""href"": ""https://datasette.io/plugins/datasette-cluster-map"", ""label"": ""datasette-cluster-map""}]" plugin_hooks:plugin-hook-extra-template-vars,plugin_hooks,plugin-hook-extra-template-vars,"extra_template_vars(template, database, table, columns, view_name, request, datasette)","Extra template variables that should be made available in the rendered template context. template - string The template that is being rendered, e.g. database.html database - string or None The name of the database, or None if the page does not correspond to a database (e.g. the root page) table - string or None The name of the table, or None if the page does not correct to a table columns - list of strings or None The names of the database columns that will be displayed on this page. None if the page does not contain a table. view_name - string The name of the view being displayed. ( index , database , table , and row are the most important ones.) request - Request object or None The current HTTP request. This can be None if the request object is not available. datasette - Datasette class You can use this to access plugin configuration options via datasette.plugin_config(your_plugin_name) This hook can return one of three different types: Dictionary If you return a dictionary its keys and values will be merged into the template context. Function that returns a dictionary If you return a function it will be executed. If it returns a dictionary those values will will be merged into the template context. Function that returns an awaitable function that returns a dictionary You can also return a function which returns an awaitable function which returns a dictionary. Datasette runs Jinja2 in async mode , which means you can add awaitable functions to the template scope and they will be automatically awaited when they are rendered by the template. Here's an example plugin that adds a ""user_agent"" variable to the template context containing the current request's User-Agent header: @hookimpl def extra_template_vars(request): return {""user_agent"": request.headers.get(""user-agent"")} This example returns an awaitable function which adds a list of hidden_table_names to the context: @hookimpl def extra_template_vars(datasette, database): async def hidden_table_names(): if database: db = datasette.databases[database] return { ""hidden_table_names"": await db.hidden_table_names() } else: return {} return hidden_table_names And here's an example which adds a sql_first(sql_query) function which executes a SQL statement and returns the first column of the first row of results: @hookimpl def extra_template_vars(datasette, database): async def sql_first(sql, dbname=None): dbname = ( dbname or database or next(iter(datasette.databases.keys())) ) result = await datasette.execute(dbname, sql) return result.rows[0][0] return {""sql_first"": sql_first} You can then use the new function in a template like so: SQLite version: {{ sql_first(""select sqlite_version()"") }} Examples: datasette-search-all , datasette-template-sql","[""Plugin hooks""]","[{""href"": ""https://jinja.palletsprojects.com/en/2.10.x/api/#async-support"", ""label"": ""async mode""}, {""href"": ""https://datasette.io/plugins/datasette-search-all"", ""label"": ""datasette-search-all""}, {""href"": ""https://datasette.io/plugins/datasette-template-sql"", ""label"": ""datasette-template-sql""}]" getting_started:getting-started-your-computer,getting_started,getting-started-your-computer,Using Datasette on your own computer,"First, follow the Installation instructions. Now you can run Datasette against a SQLite file on your computer using the following command: datasette path/to/database.db This will start a web server on port 8001 - visit http://localhost:8001/ to access the web interface. Add -o to open your browser automatically once Datasette has started: datasette path/to/database.db -o Use Chrome on OS X? You can run datasette against your browser history like so: datasette ~/Library/Application\ Support/Google/Chrome/Default/History --nolock The --nolock option ignores any file locks. This is safe as Datasette will open the file in read-only mode. Now visiting http://localhost:8001/History/downloads will show you a web interface to browse your downloads data: http://localhost:8001/History/downloads.json will return that data as JSON: { ""database"": ""History"", ""columns"": [ ""id"", ""current_path"", ""target_path"", ""start_time"", ""received_bytes"", ""total_bytes"", ... ], ""rows"": [ [ 1, ""/Users/simonw/Downloads/DropboxInstaller.dmg"", ""/Users/simonw/Downloads/DropboxInstaller.dmg"", 13097290269022132, 626688, 0, ... ] ] } http://localhost:8001/History/downloads.json?_shape=objects will return that data as JSON in a more convenient format: { ... ""rows"": [ { ""start_time"": 13097290269022132, ""interrupt_reason"": 0, ""hash"": """", ""id"": 1, ""site_url"": """", ""referrer"": ""https://www.dropbox.com/downloading?src=index"", ... } ] }","[""Getting started""]","[{""href"": ""http://localhost:8001/"", ""label"": ""http://localhost:8001/""}, {""href"": ""http://localhost:8001/History/downloads"", ""label"": ""http://localhost:8001/History/downloads""}, {""href"": ""http://localhost:8001/History/downloads.json"", ""label"": ""http://localhost:8001/History/downloads.json""}, {""href"": ""http://localhost:8001/History/downloads.json?_shape=objects"", ""label"": ""http://localhost:8001/History/downloads.json?_shape=objects""}]" changelog:id24,changelog,id24,0.59.1 (2021-10-24),"Fix compatibility with Python 3.10. ( #1482 ) Documentation on how to use Named parameters with integer and floating point values. ( #1496 )","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1482"", ""label"": ""#1482""}, {""href"": ""https://github.com/simonw/datasette/issues/1496"", ""label"": ""#1496""}]" changelog:id4,changelog,id4,0.64.4 (2023-09-21),Fix for a crashing bug caused by viewing the table page for a named in-memory database. ( #2189 ),"[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/2189"", ""label"": ""#2189""}]" changelog:id26,changelog,id26,0.58.1 (2021-07-16),Fix for an intermittent race condition caused by the refresh_schemas() internal function. ( #1231 ),"[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1231"", ""label"": ""#1231""}]" changelog:id36,changelog,id36,0.52.5 (2020-12-09),Fix for error caused by combining the _searchmode=raw and ?_search_COLUMN parameters. ( #1134 ),"[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1134"", ""label"": ""#1134""}]" changelog:id206,changelog,id206,0.10 (2017-11-14),"Fixed #83 - 500 error on individual row pages. Stop using sqlite WITH RECURSIVE in our tests. The version of Python 3 running in Travis CI doesn't support this.","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/83"", ""label"": ""#83""}]" changelog:id10,changelog,id10,0.63.2 (2022-11-18),"Fixed a bug in datasette publish heroku where deployments failed due to an older version of Python being requested. ( #1905 ) New datasette publish heroku --generate-dir option for generating a Heroku deployment directory without deploying it.","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1905"", ""label"": ""#1905""}]" changelog:id45,changelog,id45,0.50.1 (2020-10-09),"Fixed a bug introduced in 0.50 where the export as JSON/CSV links on the table, row and query pages were broken. ( #1010 )","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1010"", ""label"": ""#1010""}]" changelog:id33,changelog,id33,0.54.1 (2021-02-02),Fixed a bug where ?_search= and ?_sort= parameters were incorrectly duplicated when the filter form on the table page was re-submitted. ( #1214 ),"[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1214"", ""label"": ""#1214""}]" changelog:id9,changelog,id9,0.63.3 (2022-12-17),"Fixed a bug where datasette --root , when running in Docker, would only output the URL to sign in as root when the server shut down, not when it started up. ( #1958 ) You no longer need to ensure await datasette.invoke_startup() has been called in order for Datasette to start correctly serving requests - this is now handled automatically the first time the server receives a request. This fixes a bug experienced when Datasette is served directly by an ASGI application server such as Uvicorn or Gunicorn. It also fixes a bug with the datasette-gunicorn plugin. ( #1955 )","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1958"", ""label"": ""#1958""}, {""href"": ""https://datasette.io/plugins/datasette-gunicorn"", ""label"": ""datasette-gunicorn""}, {""href"": ""https://github.com/simonw/datasette/issues/1955"", ""label"": ""#1955""}]" changelog:id72,changelog,id72,0.31.2 (2019-11-13),"Fixed a bug where datasette publish heroku applications failed to start ( #633 ) Fix for datasette publish with just --source_url - thanks, Stanley Zheng ( #572 ) Deployments to Heroku now use Python 3.8.0 ( #632 )","[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/633"", ""label"": ""#633""}, {""href"": ""https://github.com/simonw/datasette/issues/572"", ""label"": ""#572""}, {""href"": ""https://github.com/simonw/datasette/issues/632"", ""label"": ""#632""}]" changelog:id2,changelog,id2,0.64.6 (2023-12-22),Fixed a bug where CSV export with expanded labels could fail if a foreign key reference did not correctly resolve. ( #2214 ),"[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/2214"", ""label"": ""#2214""}]" changelog:id18,changelog,id18,0.60.2 (2022-02-07),Fixed a bug where Datasette would open the same file twice with two different database names if you ran datasette file.db file.db . ( #1632 ),"[""Changelog""]","[{""href"": ""https://github.com/simonw/datasette/issues/1632"", ""label"": ""#1632""}]" changelog:id11,changelog,id11,0.63.1 (2022-11-10),"Fixed a bug where Datasette's table filter form would not redirect correctly when run behind a proxy using the base_url setting. ( #1883 ) SQL query is now shown wrapped in a