{"rowid": 101, "title": "Warning", "content": "The SpatiaLite extension adds  a large number of additional SQL functions , some of which are not be safe for untrusted users to execute: they may cause the Datasette server to crash. \n                 You should not expose a SpatiaLite-enabled Datasette instance to the public internet without taking extra measures to secure it against potentially harmful SQL queries. \n                 The following steps are recommended: \n                 \n                     \n                         Disable arbitrary SQL queries by untrusted users. See  Controlling the ability to execute arbitrary SQL  for ways to do this. The easiest is to start Datasette with the  datasette --setting default_allow_sql off  option. \n                     \n                     \n                         Define  Canned queries  with the SQL queries that use SpatiaLite functions that you want people to be able to execute. \n                     \n                 \n                 The  Datasette SpatiaLite tutorial  includes detailed instructions for running SpatiaLite safely using these techniques", "sections_fts": 1, "rank": null}
{"rowid": 102, "title": "Installation", "content": "", "sections_fts": 1, "rank": null}
{"rowid": 103, "title": "Installing SpatiaLite on OS X", "content": "The easiest way to install SpatiaLite on OS X is to use  Homebrew . \n                 brew update\nbrew install spatialite-tools \n                 This will install the  spatialite  command-line tool and the  mod_spatialite  dynamic library. \n                 You can now run Datasette like so: \n                 datasette --load-extension=spatialite", "sections_fts": 1, "rank": null}
{"rowid": 104, "title": "Installing SpatiaLite on Linux", "content": "SpatiaLite is packaged for most Linux distributions. \n                 apt install spatialite-bin libsqlite3-mod-spatialite \n                 Depending on your distribution, you should be able to run Datasette something like this: \n                 datasette --load-extension=/usr/lib/x86_64-linux-gnu/mod_spatialite.so \n                 If you are unsure of the location of the module, try running  locate mod_spatialite  and see what comes back.", "sections_fts": 1, "rank": null}
{"rowid": 105, "title": "Spatial indexing latitude/longitude columns", "content": "Here's a recipe for taking a table with existing latitude and longitude columns, adding a SpatiaLite POINT geometry column to that table, populating the new column and then populating a spatial index: \n             import sqlite3\n\nconn = sqlite3.connect(\"museums.db\")\n# Lead the spatialite extension:\nconn.enable_load_extension(True)\nconn.load_extension(\"/usr/local/lib/mod_spatialite.dylib\")\n# Initialize spatial metadata for this database:\nconn.execute(\"select InitSpatialMetadata(1)\")\n# Add a geometry column called point_geom to our museums table:\nconn.execute(\n    \"SELECT AddGeometryColumn('museums', 'point_geom', 4326, 'POINT', 2);\"\n)\n# Now update that geometry column with the lat/lon points\nconn.execute(\n    \"\"\"\n    UPDATE museums SET\n    point_geom = GeomFromText('POINT('||\"longitude\"||' '||\"latitude\"||')',4326);\n\"\"\"\n)\n# Now add a spatial index to that column\nconn.execute(\n    'select CreateSpatialIndex(\"museums\", \"point_geom\");'\n)\n# If you don't commit your changes will not be persisted:\nconn.commit()\nconn.close()", "sections_fts": 1, "rank": null}
{"rowid": 106, "title": "Making use of a spatial index", "content": "SpatiaLite spatial indexes are R*Trees. They allow you to run efficient bounding box queries using a sub-select, with a similar pattern to that used for  Searches using custom SQL . \n             In the above example, the resulting index will be called  idx_museums_point_geom . This takes the form of a SQLite virtual table. You can inspect its contents using the following query: \n             select * from idx_museums_point_geom limit 10; \n             Here's a live example:  timezones-api.datasette.io/timezones/idx_timezones_Geometry \n             \n                 \n                     \n                     \n                     \n                     \n                     \n                     \n                         \n                             \n                                 pkid \n                             \n                             \n                                 xmin \n                             \n                             \n                                 xmax \n                             \n                             \n                                 ymin \n                             \n                             \n                                 ymax \n                             \n                         \n                     \n                     \n                         \n                             \n                                 1 \n                             \n                             \n                                 -8.601725578308105 \n                             \n                             \n                                 -2.4930307865142822 \n                             \n                             \n                                 4.162120819091797 \n                             \n                             \n                                 10.74019718170166 \n                             \n                         \n                         \n                             \n                                 2 \n                             \n                             \n                                 -3.2607860565185547 \n                             \n                             \n                                 1.27329421043396 \n                             \n                             \n                                 4.539252281188965 \n                             \n                             \n                                 11.174856185913086 \n                             \n                         \n                         \n                             \n                                 3 \n                             \n                             \n                                 32.997581481933594 \n                             \n                             \n                                 47.98238754272461 \n                             \n                             \n                                 3.3974475860595703 \n                             \n                             \n                                 14.894054412841797 \n                             \n                         \n                         \n                             \n                                 4 \n                             \n                             \n                                 -8.66890811920166 \n                             \n                             \n                                 11.997337341308594 \n                             \n                             \n                                 18.9681453704834 \n                             \n                             \n                                 37.296207427978516 \n                             \n                         \n                         \n                             \n                                 5 \n                             \n                             \n                                 36.43336486816406 \n                             \n                             \n                                 43.300174713134766 \n                             \n                             \n                                 12.354820251464844 \n                             \n                             \n                                 18.070993423461914 \n                             \n                         \n                     \n                 \n             \n             You can now construct efficient bounding box queries that will make use of the index like this: \n             select * from museums where museums.rowid in (\n    SELECT pkid FROM idx_museums_point_geom\n    -- left-hand-edge of point > left-hand-edge of bbox (minx)\n    where xmin > :bbox_minx\n    -- right-hand-edge of point < right-hand-edge of bbox (maxx)\n    and xmax < :bbox_maxx\n    -- bottom-edge of point > bottom-edge of bbox (miny)\n    and ymin > :bbox_miny\n    -- top-edge of point < top-edge of bbox (maxy)\n    and ymax < :bbox_maxy\n); \n             Spatial indexes can be created against polygon columns as well as point columns, in which case they will represent the minimum bounding rectangle of that polygon. This is useful for accelerating  within  queries, as seen in the Timezones API example.", "sections_fts": 1, "rank": null}
{"rowid": 107, "title": "Importing shapefiles into SpatiaLite", "content": "The  shapefile format  is a common format for distributing geospatial data. You can use the  spatialite  command-line tool to create a new database table from a shapefile. \n             Try it now with the North America shapefile available from the University of North Carolina  Global River Database  project. Download the file and unzip it (this will create files called  narivs.dbf ,  narivs.prj ,  narivs.shp  and  narivs.shx  in the current directory), then run the following: \n             $ spatialite rivers-database.db\nSpatiaLite version ..: 4.3.0a       Supported Extensions:\n...\nspatialite> .loadshp narivs rivers CP1252 23032\n========\nLoading shapefile at 'narivs' into SQLite table 'rivers'\n...\nInserted 467973 rows into 'rivers' from SHAPEFILE \n             This will load the data from the  narivs  shapefile into a new database table called  rivers . \n             Exit out of  spatialite  (using  Ctrl+D ) and run Datasette against your new database like this: \n             datasette rivers-database.db \\\n    --load-extension=/usr/local/lib/mod_spatialite.dylib \n             If you browse to  http://localhost:8001/rivers-database/rivers  you will see the new table... but the  Geometry  column will contain unreadable binary data (SpatiaLite uses  a custom format based on WKB ). \n             The easiest way to turn this into semi-readable data is to use the SpatiaLite  AsGeoJSON  function. Try the following using the SQL query interface at  http://localhost:8001/rivers-database : \n             select *, AsGeoJSON(Geometry) from rivers limit 10; \n             This will give you back an additional column of GeoJSON. You can copy and paste GeoJSON from this column into the debugging tool at  geojson.io  to visualize it on a map. \n             To see a more interesting example, try ordering the records with the longest geometry first. Since there are 467,000 rows in the table you will first need to increase the SQL time limit imposed by Datasette: \n             datasette rivers-database.db \\\n    --load-extension=/usr/local/lib/mod_spatialite.dylib \\\n    --setting sql_time_limit_ms 10000 \n             Now try the following query: \n             select *, AsGeoJSON(Geometry) from rivers\norder by length(Geometry) desc limit 10;", "sections_fts": 1, "rank": null}
{"rowid": 108, "title": "Importing GeoJSON polygons using Shapely", "content": "Another common form of polygon data is the GeoJSON format. This can be imported into SpatiaLite directly, or by using the  Shapely  Python library. \n             Who's On First  is an excellent source of openly licensed GeoJSON polygons. Let's import the geographical polygon for Wales. First, we can use the Who's On First Spelunker tool to find the record for Wales: \n             spelunker.whosonfirst.org/id/404227475 \n             That page includes a link to the GeoJSON record, which can be accessed here: \n             data.whosonfirst.org/404/227/475/404227475.geojson \n             Here's Python code to create a SQLite database, enable SpatiaLite, create a places table and then add a record for Wales: \n             import sqlite3\n\nconn = sqlite3.connect(\"places.db\")\n# Enable SpatialLite extension\nconn.enable_load_extension(True)\nconn.load_extension(\"/usr/local/lib/mod_spatialite.dylib\")\n# Create the masic countries table\nconn.execute(\"select InitSpatialMetadata(1)\")\nconn.execute(\n    \"create table places (id integer primary key, name text);\"\n)\n# Add a MULTIPOLYGON Geometry column\nconn.execute(\n    \"SELECT AddGeometryColumn('places', 'geom', 4326, 'MULTIPOLYGON', 2);\"\n)\n# Add a spatial index against the new column\nconn.execute(\"SELECT CreateSpatialIndex('places', 'geom');\")\n# Now populate the table\nfrom shapely.geometry.multipolygon import MultiPolygon\nfrom shapely.geometry import shape\nimport requests\n\ngeojson = requests.get(\n    \"https://data.whosonfirst.org/404/227/475/404227475.geojson\"\n).json()\n# Convert to \"Well Known Text\" format\nwkt = shape(geojson[\"geometry\"]).wkt\n# Insert and commit the record\nconn.execute(\n    \"INSERT INTO places (id, name, geom) VALUES(null, ?, GeomFromText(?, 4326))\",\n    (\"Wales\", wkt),\n)\nconn.commit()", "sections_fts": 1, "rank": null}
{"rowid": 109, "title": "Querying polygons using within()", "content": "The  within()  SQL function can be used to check if a point is within a geometry: \n             select\n    name\nfrom\n    places\nwhere\n   within(GeomFromText('POINT(-3.1724366 51.4704448)'), places.geom); \n             The  GeomFromText()  function takes a string of well-known text. Note that the order used here is  longitude  then   latitude . \n             To run that same  within()  query in a way that benefits from the spatial index, use the following: \n             select\n    name\nfrom\n    places\nwhere\n    within(GeomFromText('POINT(-3.1724366 51.4704448)'), places.geom)\n    and rowid in (\n        SELECT pkid FROM idx_places_geom\n        where xmin < -3.1724366\n        and xmax > -3.1724366\n        and ymin < 51.4704448\n        and ymax > 51.4704448\n    );", "sections_fts": 1, "rank": null}
{"rowid": 110, "title": "Custom pages and templates", "content": "Datasette provides a number of ways of customizing the way data is displayed.", "sections_fts": 1, "rank": null}
{"rowid": 111, "title": "Custom CSS and JavaScript", "content": "When you launch Datasette, you can specify a custom metadata file like this: \n             datasette mydb.db --metadata metadata.json \n             Your  metadata.json  file can include links that look like this: \n             {\n    \"extra_css_urls\": [\n        \"https://simonwillison.net/static/css/all.bf8cd891642c.css\"\n    ],\n    \"extra_js_urls\": [\n        \"https://code.jquery.com/jquery-3.2.1.slim.min.js\"\n    ]\n} \n             The extra CSS and JavaScript files will be linked in the  <head>  of every page: \n             <link rel=\"stylesheet\" href=\"https://simonwillison.net/static/css/all.bf8cd891642c.css\">\n<script src=\"https://code.jquery.com/jquery-3.2.1.slim.min.js\"></script> \n             You can also specify a SRI (subresource integrity hash) for these assets: \n             {\n    \"extra_css_urls\": [\n        {\n            \"url\": \"https://simonwillison.net/static/css/all.bf8cd891642c.css\",\n            \"sri\": \"sha384-9qIZekWUyjCyDIf2YK1FRoKiPJq4PHt6tp/ulnuuyRBvazd0hG7pWbE99zvwSznI\"\n        }\n    ],\n    \"extra_js_urls\": [\n        {\n            \"url\": \"https://code.jquery.com/jquery-3.2.1.slim.min.js\",\n            \"sri\": \"sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g=\"\n        }\n    ]\n} \n             This will produce: \n             <link rel=\"stylesheet\" href=\"https://simonwillison.net/static/css/all.bf8cd891642c.css\"\n    integrity=\"sha384-9qIZekWUyjCyDIf2YK1FRoKiPJq4PHt6tp/ulnuuyRBvazd0hG7pWbE99zvwSznI\"\n    crossorigin=\"anonymous\">\n<script src=\"https://code.jquery.com/jquery-3.2.1.slim.min.js\"\n    integrity=\"sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g=\"\n    crossorigin=\"anonymous\"></script> \n             Modern browsers will only execute the stylesheet or JavaScript if the SRI hash\n                matches the content served. You can generate hashes using  www.srihash.org \n             Items in  \"extra_js_urls\"  can specify  \"module\": true  if they reference JavaScript that uses  JavaScript modules . This configuration: \n             {\n    \"extra_js_urls\": [\n        {\n            \"url\": \"https://example.datasette.io/module.js\",\n            \"module\": true\n        }\n    ]\n} \n             Will produce this HTML: \n             <script type=\"module\" src=\"https://example.datasette.io/module.js\"></script>", "sections_fts": 1, "rank": null}
{"rowid": 112, "title": "CSS classes on the <body>", "content": "Every default template includes CSS classes in the body designed to support\n                    custom styling. \n                 The index template (the top level page at  / ) gets this: \n                 <body class=\"index\"> \n                 The database template ( /dbname ) gets this: \n                 <body class=\"db db-dbname\"> \n                 The custom SQL template ( /dbname?sql=... ) gets this: \n                 <body class=\"query db-dbname\"> \n                 A canned query template ( /dbname/queryname ) gets this: \n                 <body class=\"query db-dbname query-queryname\"> \n                 The table template ( /dbname/tablename ) gets: \n                 <body class=\"table db-dbname table-tablename\"> \n                 The row template ( /dbname/tablename/rowid ) gets: \n                 <body class=\"row db-dbname table-tablename\"> \n                 The  db-x  and  table-x  classes use the database or table names themselves if\n                    they are valid CSS identifiers. If they aren't, we strip any invalid\n                    characters out and append a 6 character md5 digest of the original name, in\n                    order to ensure that multiple tables which resolve to the same stripped\n                    character version still have different CSS classes. \n                 Some examples: \n                 \"simple\" => \"simple\"\n\"MixedCase\" => \"MixedCase\"\n\"-no-leading-hyphens\" => \"no-leading-hyphens-65bea6\"\n\"_no-leading-underscores\" => \"no-leading-underscores-b921bc\"\n\"no spaces\" => \"no-spaces-7088d7\"\n\"-\" => \"336d5e\"\n\"no $ characters\" => \"no--characters-59e024\" \n                 <td>  and  <th>  elements also get custom CSS classes reflecting the\n                    database column they are representing, for example: \n                 <table>\n    <thead>\n        <tr>\n            <th class=\"col-id\" scope=\"col\">id</th>\n            <th class=\"col-name\" scope=\"col\">name</th>\n        </tr>\n    </thead>\n    <tbody>\n        <tr>\n            <td class=\"col-id\"><a href=\"...\">1</a></td>\n            <td class=\"col-name\">SMITH</td>\n        </tr>\n    </tbody>\n</table>", "sections_fts": 1, "rank": null}
{"rowid": 113, "title": "Serving static files", "content": "Datasette can serve static files for you, using the  --static  option.\n                    Consider the following directory structure: \n                 metadata.json\nstatic-files/styles.css\nstatic-files/app.js \n                 You can start Datasette using  --static assets:static-files/  to serve those\n                    files from the  /assets/  mount point: \n                 $ datasette -m metadata.json --static assets:static-files/ --memory \n                 The following URLs will now serve the content from those CSS and JS files: \n                 http://localhost:8001/assets/styles.css\nhttp://localhost:8001/assets/app.js \n                 You can reference those files from  metadata.json  like so: \n                 {\n    \"extra_css_urls\": [\n        \"/assets/styles.css\"\n    ],\n    \"extra_js_urls\": [\n        \"/assets/app.js\"\n    ]\n}", "sections_fts": 1, "rank": null}
{"rowid": 114, "title": "Publishing static assets", "content": "The  datasette publish  command can be used to publish your static assets,\n                    using the same syntax as above: \n                 $ datasette publish cloudrun mydb.db --static assets:static-files/ \n                 This will upload the contents of the  static-files/  directory as part of the\n                    deployment, and configure Datasette to correctly serve the assets from  /assets/ .", "sections_fts": 1, "rank": null}
{"rowid": 115, "title": "Custom templates", "content": "By default, Datasette uses default templates that ship with the package. \n             You can over-ride these templates by specifying a custom  --template-dir  like\n                this: \n             datasette mydb.db --template-dir=mytemplates/ \n             Datasette will now first look for templates in that directory, and fall back on\n                the defaults if no matches are found. \n             It is also possible to over-ride templates on a per-database, per-row or per-\n                table basis. \n             The lookup rules Datasette uses are as follows: \n             Index page (/):\n    index.html\n\nDatabase page (/mydatabase):\n    database-mydatabase.html\n    database.html\n\nCustom query page (/mydatabase?sql=...):\n    query-mydatabase.html\n    query.html\n\nCanned query page (/mydatabase/canned-query):\n    query-mydatabase-canned-query.html\n    query-mydatabase.html\n    query.html\n\nTable page (/mydatabase/mytable):\n    table-mydatabase-mytable.html\n    table.html\n\nRow page (/mydatabase/mytable/id):\n    row-mydatabase-mytable.html\n    row.html\n\nTable of rows and columns include on table page:\n    _table-table-mydatabase-mytable.html\n    _table-mydatabase-mytable.html\n    _table.html\n\nTable of rows and columns include on row page:\n    _table-row-mydatabase-mytable.html\n    _table-mydatabase-mytable.html\n    _table.html \n             If a table name has spaces or other unexpected characters in it, the template\n                filename will follow the same rules as our custom  <body>  CSS classes - for\n                example, a table called \"Food Trucks\" will attempt to load the following\n                templates: \n             table-mydatabase-Food-Trucks-399138.html\ntable.html \n             You can find out which templates were considered for a specific page by viewing\n                source on that page and looking for an HTML comment at the bottom. The comment\n                will look something like this: \n             <!-- Templates considered: *query-mydb-tz.html, query-mydb.html, query.html --> \n             This example is from the canned query page for a query called \"tz\" in the\n                database called \"mydb\". The asterisk shows which template was selected - so in\n                this case, Datasette found a template file called  query-mydb-tz.html  and\n                used that - but if that template had not been found, it would have tried for\n                 query-mydb.html  or the default  query.html . \n             It is possible to extend the default templates using Jinja template\n                inheritance. If you want to customize EVERY row template with some additional\n                content you can do so by creating a  row.html  template like this: \n             {% extends \"default:row.html\" %}\n\n{% block content %}\n<h1>EXTRA HTML AT THE TOP OF THE CONTENT BLOCK</h1>\n<p>This line renders the original block:</p>\n{{ super() }}\n{% endblock %} \n             Note the  default:row.html  template name, which ensures Jinja will inherit\n                from the default template. \n             The  _table.html  template is included by both the row and the table pages,\n                and a list of rows. The default  _table.html  template renders them as an\n                HTML template and  can be seen here . \n             You can provide a custom template that applies to all of your databases and\n                tables, or you can provide custom templates for specific tables using the\n                template naming scheme described above. \n             If you want to present your data in a format other than an HTML table, you\n                can do so by looping through  display_rows  in your own  _table.html \n                template. You can use  {{ row[\"column_name\"] }}  to output the raw value\n                of a specific column. \n             If you want to output the rendered HTML version of a column, including any\n                links to foreign keys, you can use  {{ row.display(\"column_name\") }} . \n             Here is an example of a custom  _table.html  template: \n             {% for row in display_rows %}\n    <div>\n        <h2>{{ row[\"title\"] }}</h2>\n        <p>{{ row[\"description\"] }}<lp>\n        <p>Category: {{ row.display(\"category_id\") }}</p>\n    </div>\n{% endfor %}", "sections_fts": 1, "rank": null}
{"rowid": 116, "title": "Custom pages", "content": "You can add templated pages to your Datasette instance by creating HTML files in a  pages  directory within your  templates  directory. \n             For example, to add a custom page that is served at  http://localhost/about  you would create a file in  templates/pages/about.html , then start Datasette like this: \n             $ datasette mydb.db --template-dir=templates/ \n             You can nest directories within pages to create a nested structure. To create a  http://localhost:8001/about/map  page you would create  templates/pages/about/map.html .", "sections_fts": 1, "rank": null}
{"rowid": 117, "title": "Path parameters for pages", "content": "You can define custom pages that match multiple paths by creating files with  {variable}  definitions in their filenames. \n                 For example, to capture any request to a URL matching  /about/* , you would create a template in the following location: \n                 templates/pages/about/{slug}.html \n                 A hit to  /about/news  would render that template and pass in a variable called  slug  with a value of  \"news\" . \n                 If you use this mechanism don't forget to return a 404 if the referenced content could not be found. You can do this using  {{ raise_404() }}  described below. \n                 Templates defined using custom page routes work particularly well with the  sql()  template function from  datasette-template-sql  or the  graphql()  template function from  datasette-graphql .", "sections_fts": 1, "rank": null}
{"rowid": 118, "title": "Custom headers and status codes", "content": "Custom pages default to being served with a content-type of  text/html; charset=utf-8  and a  200  status code. You can change these by calling a custom function from within your template. \n                 For example, to serve a custom page with a  418 I'm a teapot  HTTP status code, create a file in  pages/teapot.html  containing the following: \n                 {{ custom_status(418) }}\n<html>\n<head><title>Teapot</title></head>\n<body>\nI'm a teapot\n</body>\n</html> \n                 To serve a custom HTTP header, add a  custom_header(name, value)  function call. For example: \n                 {{ custom_status(418) }}\n{{ custom_header(\"x-teapot\", \"I am\") }}\n<html>\n<head><title>Teapot</title></head>\n<body>\nI'm a teapot\n</body>\n</html> \n                 You can verify this is working using  curl  like this: \n                 $ curl -I 'http://127.0.0.1:8001/teapot'\nHTTP/1.1 418\ndate: Sun, 26 Apr 2020 18:38:30 GMT\nserver: uvicorn\nx-teapot: I am\ncontent-type: text/html; charset=utf-8", "sections_fts": 1, "rank": null}
{"rowid": 119, "title": "Returning 404s", "content": "To indicate that content could not be found and display the default 404 page you can use the  raise_404(message)  function: \n                 {% if not rows %}\n    {{ raise_404(\"Content not found\") }}\n{% endif %} \n                 If you call  raise_404()  the other content in your template will be ignored.", "sections_fts": 1, "rank": null}
{"rowid": 120, "title": "Custom redirects", "content": "You can use the  custom_redirect(location)  function to redirect users to another page, for example in a file called  pages/datasette.html : \n                 {{ custom_redirect(\"https://github.com/simonw/datasette\") }} \n                 Now requests to  http://localhost:8001/datasette  will result in a redirect. \n                 These redirects are served with a  302 Found  status code by default. You can send a  301 Moved Permanently  code by passing  301  as the second argument to the function: \n                 {{ custom_redirect(\"https://github.com/simonw/datasette\", 301) }}", "sections_fts": 1, "rank": null}
{"rowid": 121, "title": "Custom error pages", "content": "Datasette returns an error page if an unexpected error occurs, access is forbidden or content cannot be found. \n             You can customize the response returned for these errors by providing a custom error page template. \n             Content not found errors use a  404.html  template. Access denied errors use  403.html . Invalid input errors use  400.html . Unexpected errors of other kinds use  500.html . \n             If a template for the specific error code is not found a template called  error.html  will be used instead. If you do not provide that template Datasette's  default error.html template  will be used. \n             The error template will be passed the following context: \n             \n                 \n                     status  - integer \n                     \n                         The integer HTTP status code, e.g. 404, 500, 403, 400. \n                     \n                 \n                 \n                     error  - string \n                     \n                         Details of the specific error, usually a full sentence. \n                     \n                 \n                 \n                     title  - string or None \n                     \n                         A title for the page representing the class of error. This is often  None  for errors that do not provide a title separate from their  error  message.", "sections_fts": 1, "rank": null}
{"rowid": 122, "title": "Full-text search", "content": "SQLite includes  a powerful mechanism for enabling full-text search  against SQLite records. Datasette can detect if a table has had full-text search configured for it in the underlying database and display a search interface for filtering that table. \n         Here's  an example search : \n         \n         Datasette automatically detects which tables have been configured for full-text search.", "sections_fts": 1, "rank": null}
{"rowid": 123, "title": "The table page and table view API", "content": "Table views that support full-text search can be queried using the  ?_search=TERMS  query string parameter. This will run the search against content from all of the columns that have been included in the index. \n             Try this example:  fara.datasettes.com/fara/FARA_All_ShortForms?_search=manafort \n             SQLite full-text search supports wildcards. This means you can easily implement prefix auto-complete by including an asterisk at the end of the search term - for example: \n             /dbname/tablename/?_search=rob* \n             This will return all records containing at least one word that starts with the letters  rob . \n             You can also run searches against just the content of a specific named column by using  _search_COLNAME=TERMS  - for example, this would search for just rows where the  name  column in the FTS index mentions  Sarah : \n             /dbname/tablename/?_search_name=Sarah", "sections_fts": 1, "rank": null}
{"rowid": 124, "title": "Advanced SQLite search queries", "content": "SQLite full-text search includes support for  a variety of advanced queries , including  AND ,  OR ,  NOT  and  NEAR . \n             By default Datasette disables these features to ensure they do not cause errors or confusion for users who are not aware of them. You can disable this escaping and use the advanced queries by adding  &_searchmode=raw  to the table page query string. \n             If you want to enable these operators by default for a specific table, you can do so by adding  \"searchmode\": \"raw\"  to the metadata configuration for that table, see  Configuring full-text search for a table or view . \n             If that option has been specified in the table metadata but you want to over-ride it and return to the default behavior you can append  &_searchmode=escaped  to the query string.", "sections_fts": 1, "rank": null}
{"rowid": 125, "title": "Configuring full-text search for a table or view", "content": "If a table has a corresponding FTS table set up using the  content=  argument to  CREATE VIRTUAL TABLE  shown below, Datasette will detect it automatically and add a search interface to the table page for that table. \n             You can also manually configure which table should be used for full-text search using query string parameters or  Metadata . You can set the associated FTS table for a specific table and you can also set one for a view - if you do that, the page for that SQL view will offer a search option. \n             Use  ?_fts_table=x  to over-ride the FTS table for a specific page. If the primary key was something other than  rowid  you can use  ?_fts_pk=col  to set that as well. This is particularly useful for views, for example: \n             https://latest.datasette.io/fixtures/searchable_view?_fts_table=searchable_fts&_fts_pk=pk \n             The  fts_table  metadata property can be used to specify an associated FTS table. If the primary key column in your table which was used to populate the FTS table is something other than  rowid , you can specify the column to use with the  fts_pk  property. \n             The  \"searchmode\": \"raw\"  property can be used to default the table to accepting SQLite advanced search operators, as described in  Advanced SQLite search queries . \n             Here is an example which enables full-text search (with SQLite advanced search operators) for a  display_ads  view which is defined against the  ads  table and hence needs to run FTS against the  ads_fts  table, using the  id  as the primary key: \n             {\n    \"databases\": {\n        \"russian-ads\": {\n            \"tables\": {\n                \"display_ads\": {\n                    \"fts_table\": \"ads_fts\",\n                    \"fts_pk\": \"id\",\n                    \"searchmode\": \"raw\"\n                }\n            }\n        }\n    }\n}", "sections_fts": 1, "rank": null}
{"rowid": 126, "title": "Searches using custom SQL", "content": "You can include full-text search results in custom SQL queries. The general pattern with SQLite search is to run the search as a sub-select that returns rowid values, then include those rowids in another part of the query. \n             You can see the syntax for a basic search by running that search on a table page and then clicking \"View and edit SQL\" to see the underlying SQL. For example, consider this search for  manafort is the US FARA database : \n             /fara/FARA_All_ShortForms?_search=manafort \n             If you click  View and edit SQL  you'll see that the underlying SQL looks like this: \n             select\n  rowid,\n  Short_Form_Termination_Date,\n  Short_Form_Date,\n  Short_Form_Last_Name,\n  Short_Form_First_Name,\n  Registration_Number,\n  Registration_Date,\n  Registrant_Name,\n  Address_1,\n  Address_2,\n  City,\n  State,\n  Zip\nfrom\n  FARA_All_ShortForms\nwhere\n  rowid in (\n    select\n      rowid\n    from\n      FARA_All_ShortForms_fts\n    where\n      FARA_All_ShortForms_fts match escape_fts(:search)\n  )\norder by\n  rowid\nlimit\n  101", "sections_fts": 1, "rank": null}
{"rowid": 127, "title": "Enabling full-text search for a SQLite table", "content": "Datasette takes advantage of the  external content  mechanism in SQLite, which allows a full-text search virtual table to be associated with the contents of another SQLite table. \n             To set up full-text search for a table, you need to do two things: \n             \n                 \n                     Create a new FTS virtual table associated with your table \n                 \n                 \n                     Populate that FTS table with the data that you would like to be able to run searches against", "sections_fts": 1, "rank": null}
{"rowid": 128, "title": "Configuring FTS using sqlite-utils", "content": "sqlite-utils  is a CLI utility and Python library for manipulating SQLite databases. You can use  it from Python code  to configure FTS search, or you can achieve the same goal  using the accompanying command-line tool . \n                 Here's how to use  sqlite-utils  to enable full-text search for an  items  table across the  name  and  description  columns: \n                 $ sqlite-utils enable-fts mydatabase.db items name description", "sections_fts": 1, "rank": null}
{"rowid": 129, "title": "Configuring FTS using csvs-to-sqlite", "content": "If your data starts out in CSV files, you can use Datasette's companion tool  csvs-to-sqlite  to convert that file into a SQLite database and enable full-text search on specific columns. For a file called  items.csv  where you want full-text search to operate against the  name  and  description  columns you would run the following: \n                 $ csvs-to-sqlite items.csv items.db -f name -f description", "sections_fts": 1, "rank": null}
{"rowid": 130, "title": "Configuring FTS by hand", "content": "We recommend using  sqlite-utils , but if you want to hand-roll a SQLite full-text search table you can do so using the following SQL. \n                 To enable full-text search for a table called  items  that works against the  name  and  description  columns, you would run this SQL to create a new  items_fts  FTS virtual table: \n                 CREATE VIRTUAL TABLE \"items_fts\" USING FTS4 (\n    name,\n    description,\n    content=\"items\"\n); \n                 This creates a set of tables to power full-text search against  items . The new  items_fts  table will be detected by Datasette as the  fts_table  for the  items  table. \n                 Creating the table is not enough: you also need to populate it with a copy of the data that you wish to make searchable. You can do that using the following SQL: \n                 INSERT INTO \"items_fts\" (rowid, name, description)\n    SELECT rowid, name, description FROM items; \n                 If your table has columns that are foreign key references to other tables you can include that data in your full-text search index using a join. Imagine the  items  table has a foreign key column called  category_id  which refers to a  categories  table - you could create a full-text search table like this: \n                 CREATE VIRTUAL TABLE \"items_fts\" USING FTS4 (\n    name,\n    description,\n    category_name,\n    content=\"items\"\n); \n                 And then populate it like this: \n                 INSERT INTO \"items_fts\" (rowid, name, description, category_name)\n    SELECT items.rowid,\n    items.name,\n    items.description,\n    categories.name\n    FROM items JOIN categories ON items.category_id=categories.id; \n                 You can use this technique to populate the full-text search index from any combination of tables and joins that makes sense for your project.", "sections_fts": 1, "rank": null}
{"rowid": 131, "title": "FTS versions", "content": "There are three different versions of the SQLite FTS module: FTS3, FTS4 and FTS5. You can tell which versions are supported by your instance of Datasette by checking the  /-/versions  page. \n             FTS5 is the most advanced module but may not be available in the SQLite version that is bundled with your Python installation. Most importantly, FTS5 is the only version that has the ability to order by search relevance without needing extra code. \n             If you can't be sure that FTS5 will be available, you should use FTS4.", "sections_fts": 1, "rank": null}
{"rowid": 132, "title": "Contributing", "content": "Datasette is an open source project. We welcome contributions! \n         This document describes how to contribute to Datasette core. You can also contribute to the wider Datasette ecosystem by creating new  Plugins .", "sections_fts": 1, "rank": null}
{"rowid": 133, "title": "General guidelines", "content": "main should always be releasable . Incomplete features should live in branches. This ensures that any small bug fixes can be quickly released. \n                 \n                 \n                     The ideal commit  should bundle together the implementation, unit tests and associated documentation updates. The commit message should link to an associated issue. \n                 \n                 \n                     New plugin hooks  should only be shipped if accompanied by a separate release of a non-demo plugin that uses them.", "sections_fts": 1, "rank": null}
{"rowid": 134, "title": "Setting up a development environment", "content": "If you have Python 3.7 or higher installed on your computer (on OS X the quickest way to do this  is using homebrew ) you can install an editable copy of Datasette using the following steps. \n             If you want to use GitHub to publish your changes, first  create a fork of datasette  under your own GitHub account. \n             Now clone that repository somewhere on your computer: \n             git clone git@github.com:YOURNAME/datasette \n             If you want to get started without creating your own fork, you can do this instead: \n             git clone git@github.com:simonw/datasette \n             The next step is to create a virtual environment for your project and use it to install Datasette's dependencies: \n             cd datasette\n# Create a virtual environment in ./venv\npython3 -m venv ./venv\n# Now activate the virtual environment, so pip can install into it\nsource venv/bin/activate\n# Install Datasette and its testing dependencies\npython3 -m pip install -e '.[test]' \n             That last line does most of the work:  pip install -e  means \"install this package in a way that allows me to edit the source code in place\". The  .[test]  option means \"use the setup.py in this directory and install the optional testing dependencies as well\".", "sections_fts": 1, "rank": null}
{"rowid": 135, "title": "Running the tests", "content": "Once you have done this, you can run the Datasette unit tests from inside your  datasette/  directory using  pytest  like so: \n             pytest \n             You can run the tests faster using multiple CPU cores with  pytest-xdist  like this: \n             pytest -n auto -m \"not serial\" \n             -n auto  detects the number of available cores automatically. The  -m \"not serial\"  skips tests that don't work well in a parallel test environment. You can run those tests separately like so: \n             pytest -m \"serial\"", "sections_fts": 1, "rank": null}
{"rowid": 136, "title": "Using fixtures", "content": "To run Datasette itself, type  datasette . \n             You're going to need at least one SQLite database. A quick way to get started is to use the fixtures database that Datasette uses for its own tests. \n             You can create a copy of that database by running this command: \n             python tests/fixtures.py fixtures.db \n             Now you can run Datasette against the new fixtures database like so: \n             datasette fixtures.db \n             This will start a server at  http://127.0.0.1:8001/ . \n             Any changes you make in the  datasette/templates  or  datasette/static  folder will be picked up immediately (though you may need to do a force-refresh in your browser to see changes to CSS or JavaScript). \n             If you want to change Datasette's Python code you can use the  --reload  option to cause Datasette to automatically reload any time the underlying code changes: \n             datasette --reload fixtures.db \n             You can also use the  fixtures.py  script to recreate the testing version of  metadata.json  used by the unit tests. To do that: \n             python tests/fixtures.py fixtures.db fixtures-metadata.json \n             Or to output the plugins used by the tests, run this: \n             python tests/fixtures.py fixtures.db fixtures-metadata.json fixtures-plugins\nTest tables written to fixtures.db\n- metadata written to fixtures-metadata.json\nWrote plugin: fixtures-plugins/register_output_renderer.py\nWrote plugin: fixtures-plugins/view_name.py\nWrote plugin: fixtures-plugins/my_plugin.py\nWrote plugin: fixtures-plugins/messages_output_renderer.py\nWrote plugin: fixtures-plugins/my_plugin_2.py \n             Then run Datasette like this: \n             datasette fixtures.db -m fixtures-metadata.json --plugins-dir=fixtures-plugins/", "sections_fts": 1, "rank": null}
{"rowid": 137, "title": "Debugging", "content": "Any errors that occur while Datasette is running while display a stack trace on the console. \n             You can tell Datasette to open an interactive  pdb  debugger session if an error occurs using the  --pdb  option: \n             datasette --pdb fixtures.db", "sections_fts": 1, "rank": null}
{"rowid": 138, "title": "Code formatting", "content": "Datasette uses opinionated code formatters:  Black  for Python and  Prettier  for JavaScript. \n             These formatters are enforced by Datasette's continuous integration: if a commit includes Python or JavaScript code that does not match the style enforced by those tools, the tests will fail. \n             When developing locally, you can verify and correct the formatting of your code using these tools.", "sections_fts": 1, "rank": null}
{"rowid": 139, "title": "Running Black", "content": "Black will be installed when you run  pip install -e '.[test]' . To test that your code complies with Black, run the following in your root  datasette  repository checkout: \n                 $ black . --check\nAll done! \u2728 \ud83c\udf70 \u2728\n95 files would be left unchanged. \n                 If any of your code does not conform to Black you can run this to automatically fix those problems: \n                 $ black .\nreformatted ../datasette/setup.py\nAll done! \u2728 \ud83c\udf70 \u2728\n1 file reformatted, 94 files left unchanged.", "sections_fts": 1, "rank": null}
{"rowid": 140, "title": "blacken-docs", "content": "The  blacken-docs  command applies Black formatting rules to code examples in the documentation. Run it like this: \n                 blacken-docs -l 60 docs/*.rst", "sections_fts": 1, "rank": null}
{"rowid": 141, "title": "Prettier", "content": "To install Prettier,  install Node.js  and then run the following in the root of your  datasette  repository checkout: \n                 $ npm install \n                 This will install Prettier in a  node_modules  directory. You can then check that your code matches the coding style like so: \n                 $ npm run prettier -- --check\n> prettier\n> prettier 'datasette/static/*[!.min].js' \"--check\"\n\nChecking formatting...\n[warn] datasette/static/plugins.js\n[warn] Code style issues found in the above file(s). Forgot to run Prettier? \n                 You can fix any problems by running: \n                 $ npm run fix", "sections_fts": 1, "rank": null}
{"rowid": 142, "title": "Editing and building the documentation", "content": "Datasette's documentation lives in the  docs/  directory and is deployed automatically using  Read The Docs . \n             The documentation is written using reStructuredText. You may find this article on  The subset of reStructuredText worth committing to memory  useful. \n             You can build it locally by installing  sphinx  and  sphinx_rtd_theme  in your Datasette development environment and then running  make html  directly in the  docs/  directory: \n             # You may first need to activate your virtual environment:\nsource venv/bin/activate\n\n# Install the dependencies needed to build the docs\npip install -e .[docs]\n\n# Now build the docs\ncd docs/\nmake html \n             This will create the HTML version of the documentation in  docs/_build/html . You can open it in your browser like so: \n             open _build/html/index.html \n             Any time you make changes to a  .rst  file you can re-run  make html  to update the built documents, then refresh them in your browser. \n             For added productivity, you can use use  sphinx-autobuild  to run Sphinx in auto-build mode. This will run a local webserver serving the docs that automatically rebuilds them and refreshes the page any time you hit save in your editor. \n             sphinx-autobuild  will have been installed when you ran  pip install -e .[docs] . In your  docs/  directory you can start the server by running the following: \n             make livehtml \n             Now browse to  http://localhost:8000/  to view the documentation. Any edits you make should be instantly reflected in your browser.", "sections_fts": 1, "rank": null}
{"rowid": 143, "title": "Running Cog", "content": "Some pages of documentation (in particular the  CLI reference ) are automatically updated using  Cog . \n                 To update these pages, run the following command: \n                 cog -r docs/*.rst", "sections_fts": 1, "rank": null}
{"rowid": 144, "title": "Continuously deployed demo instances", "content": "The demo instance at  latest.datasette.io  is re-deployed automatically to Google Cloud Run for every push to  main  that passes the test suite. This is implemented by the GitHub Actions workflow at  .github/workflows/deploy-latest.yml . \n             Specific branches can also be set to automatically deploy by adding them to the  on: push: branches  block at the top of the workflow YAML file. Branches configured in this way will be deployed to a new Cloud Run service whether or not their tests pass. \n             The Cloud Run URL for a branch demo can be found in the GitHub Actions logs.", "sections_fts": 1, "rank": null}
{"rowid": 145, "title": "Release process", "content": "Datasette releases are performed using tags. When a new release is published on GitHub, a  GitHub Action workflow  will perform the following: \n             \n                 \n                     Run the unit tests against all supported Python versions. If the tests pass... \n                 \n                 \n                     Build a Docker image of the release and push a tag to  https://hub.docker.com/r/datasetteproject/datasette \n                 \n                 \n                     Re-point the \"latest\" tag on Docker Hub to the new image \n                 \n                 \n                     Build a wheel bundle of the underlying Python source code \n                 \n                 \n                     Push that new wheel up to PyPI:  https://pypi.org/project/datasette/ \n                 \n             \n             To deploy new releases you will need to have push access to the main Datasette GitHub repository. \n             Datasette follows  Semantic Versioning : \n             major.minor.patch \n             We increment  major  for backwards-incompatible releases. Datasette is currently pre-1.0 so the major version is always  0 . \n             We increment  minor  for new features. \n             We increment  patch  for bugfix releass. \n             Alpha and beta releases  may have an additional  a0  or  b0  prefix - the integer component will be incremented with each subsequent alpha or beta. \n             To release a new version, first create a commit that updates the version number in  datasette/version.py  and the  the changelog  with highlights of the new version. An example  commit can be seen here : \n             # Update changelog\ngit commit -m \" Release 0.51a1\n\nRefs #1056, #1039, #998, #1045, #1033, #1036, #1034, #976, #1057, #1058, #1053, #1064, #1066\" -a\ngit push \n             Referencing the issues that are part of the release in the commit message ensures the name of the release shows up on those issue pages, e.g.  here . \n             You can generate the list of issue references for a specific release by copying and pasting text from the release notes or GitHub changes-since-last-release view into this  Extract issue numbers from pasted text  tool. \n             To create the tag for the release, create  a new release  on GitHub matching the new version number. You can convert the release notes to Markdown by copying and pasting the rendered HTML into this  Paste to Markdown tool . \n             Finally, post a news item about the release on  datasette.io  by editing the  news.yaml  file in that site's repository.", "sections_fts": 1, "rank": null}
{"rowid": 146, "title": "Alpha and beta releases", "content": "Alpha and beta releases are published to preview upcoming features that may not yet be stable - in particular to preview new plugin hooks. \n             You are welcome to try these out, but please be aware that details may change before the final release. \n             Please join  discussions on the issue tracker  to share your thoughts and experiences with on alpha and beta features that you try out.", "sections_fts": 1, "rank": null}
{"rowid": 147, "title": "Releasing bug fixes from a branch", "content": "If it's necessary to publish a bug fix release without shipping new features that have landed on  main  a release branch can be used. \n             Create it from the relevant last tagged release like so: \n             git branch 0.52.x 0.52.4\ngit checkout 0.52.x \n             Next cherry-pick the commits containing the bug fixes: \n             git cherry-pick COMMIT \n             Write the release notes in the branch, and update the version number in  version.py . Then push the branch: \n             git push -u origin 0.52.x \n             Once the tests have completed, publish the release from that branch target using the GitHub  Draft a new release  form. \n             Finally, cherry-pick the commit with the release notes and version number bump across to  main : \n             git checkout main\ngit cherry-pick COMMIT\ngit push", "sections_fts": 1, "rank": null}
{"rowid": 148, "title": "Upgrading CodeMirror", "content": "Datasette bundles  CodeMirror  for the SQL editing interface, e.g. on  this page . Here are the steps for upgrading to a new version of CodeMirror: \n             \n                 \n                     Download and extract latest CodeMirror zip file from  https://codemirror.net/codemirror.zip \n                 \n                 \n                     Rename  lib/codemirror.js  to  codemirror-5.57.0.js  (using latest version number) \n                 \n                 \n                     Rename  lib/codemirror.css  to  codemirror-5.57.0.css \n                 \n                 \n                     Rename  mode/sql/sql.js  to  codemirror-5.57.0-sql.js \n                 \n                 \n                     Edit both JavaScript files to make the top license comment a  /* */  block instead of multiple  //  lines \n                 \n                 \n                     Minify the JavaScript files like this: \n                     npx uglify-js codemirror-5.57.0.js -o codemirror-5.57.0.min.js --comments '/LICENSE/'\nnpx uglify-js codemirror-5.57.0-sql.js -o codemirror-5.57.0-sql.min.js --comments '/LICENSE/' \n                 \n                 \n                     Check that the LICENSE comment did indeed survive minification \n                 \n                 \n                     Minify the CSS file like this: \n                     npx clean-css-cli codemirror-5.57.0.css -o codemirror-5.57.0.min.css \n                 \n                 \n                     Edit the  _codemirror.html  template to reference the new files \n                 \n                 \n                     git rm  the old files,  git add  the new files", "sections_fts": 1, "rank": null}
{"rowid": 149, "title": "Pages and API endpoints", "content": "The Datasette web application offers a number of different pages that can be accessed to explore the data in question, each of which is accompanied by an equivalent JSON API.", "sections_fts": 1, "rank": null}
{"rowid": 150, "title": "Top-level index", "content": "The root page of any Datasette installation is an index page that lists all of the currently attached databases. Some examples: \n             \n                 \n                     fivethirtyeight.datasettes.com \n                 \n                 \n                     global-power-plants.datasettes.com \n                 \n                 \n                     register-of-members-interests.datasettes.com \n                 \n             \n             Add  /.json  to the end of the URL for the JSON version of the underlying data: \n             \n                 \n                     fivethirtyeight.datasettes.com/.json \n                 \n                 \n                     global-power-plants.datasettes.com/.json \n                 \n                 \n                     register-of-members-interests.datasettes.com/.json", "sections_fts": 1, "rank": null}
{"rowid": 151, "title": "Database", "content": "Each database has a page listing the tables, views and canned queries available for that database. If the  execute-sql  permission is enabled (it's on by default) there will also be an interface for executing arbitrary SQL select queries against the data. \n             Examples: \n             \n                 \n                     fivethirtyeight.datasettes.com/fivethirtyeight \n                 \n                 \n                     global-power-plants.datasettes.com/global-power-plants \n                 \n             \n             The JSON version of this page provides programmatic access to the underlying data: \n             \n                 \n                     fivethirtyeight.datasettes.com/fivethirtyeight.json \n                 \n                 \n                     global-power-plants.datasettes.com/global-power-plants.json", "sections_fts": 1, "rank": null}
{"rowid": 152, "title": "Table", "content": "The table page is the heart of Datasette: it allows users to interactively explore the contents of a database table, including sorting, filtering,  Full-text search  and applying  Facets . \n             The HTML interface is worth spending some time exploring. As with other pages, you can return the JSON data by appending  .json  to the URL path, before any  ?  query string arguments. \n             The query string arguments are described in more detail here:  Table arguments \n             You can also use the table page to interactively construct a SQL query - by applying different filters and a sort order for example - and then click the \"View and edit SQL\" link to see the SQL query that was used for the page and edit and re-submit it. \n             Some examples: \n             \n                 \n                     ../items  lists all of the line-items registered by UK MPs as potential conflicts of interest. It demonstrates Datasette's support for  Full-text search . \n                 \n                 \n                     ../antiquities-act%2Factions_under_antiquities_act  is an interface for exploring the \"actions under the antiquities act\" data table published by FiveThirtyEight. \n                 \n                 \n                     ../global-power-plants?country_long=United+Kingdom&primary_fuel=Gas  is a filtered table page showing every Gas power plant in the United Kingdom. It includes some default facets (configured using  its metadata.json ) and uses the  datasette-cluster-map  plugin to show a map of the results.", "sections_fts": 1, "rank": null}
{"rowid": 153, "title": "Row", "content": "Every row in every Datasette table has its own URL. This means individual records can be linked to directly. \n             Table cells with extremely long text contents are truncated on the table view according to the  truncate_cells_html  setting. If a cell has been truncated the full length version of that cell will be available on the row page. \n             Rows which are the targets of foreign key references from other tables will show a link to a filtered search for all records that reference that row. Here's an example from the Registers of Members Interests database: \n             ../people/uk.org.publicwhip%2Fperson%2F10001 \n             Note that this URL includes the encoded primary key of the record. \n             Here's that same page as JSON: \n             ../people/uk.org.publicwhip%2Fperson%2F10001.json", "sections_fts": 1, "rank": null}
{"rowid": 154, "title": "The Datasette Ecosystem", "content": "Datasette sits at the center of a growing ecosystem of open source tools aimed at making it as easy as possible to gather, analyze and publish interesting data. \n         These tools are divided into two main groups: tools for building SQLite databases (for use with Datasette) and plugins that extend Datasette's functionality. \n         The  Datasette project website  includes a directory of plugins and a directory of tools: \n         \n             \n                 Plugins directory on datasette.io \n             \n             \n                 Tools directory on datasette.io", "sections_fts": 1, "rank": null}
{"rowid": 155, "title": "sqlite-utils", "content": "sqlite-utils  is a key building block for the wider Datasette ecosystem. It provides a collection of utilities for manipulating SQLite databases, both as a Python library and a command-line utility. Features include: \n             \n                 \n                     Insert data into a SQLite database from JSON, CSV or TSV, automatically creating tables with the correct schema or altering existing tables to add missing columns. \n                 \n                 \n                     Configure tables for use with SQLite full-text search, including creating triggers needed to keep the search index up-to-date. \n                 \n                 \n                     Modify tables in ways that are not supported by SQLite's default  ALTER TABLE  syntax - for example changing the types of columns or selecting a new primary key for a table. \n                 \n                 \n                     Adding foreign keys to existing database tables. \n                 \n                 \n                     Extracting columns of data into a separate lookup table.", "sections_fts": 1, "rank": null}
{"rowid": 156, "title": "Dogsheep", "content": "Dogsheep  is a collection of tools for personal analytics using SQLite and Datasette. The project provides tools like  github-to-sqlite  and  twitter-to-sqlite  that can import data from different sources in order to create a personal data warehouse.  Personal Data Warehouses: Reclaiming Your Data  is a talk that explains Dogsheep and demonstrates it in action.", "sections_fts": 1, "rank": null}
{"rowid": 157, "title": "Introspection", "content": "Datasette includes some pages and JSON API endpoints for introspecting the current instance. These can be used to understand some of the internals of Datasette and to see how a particular instance has been configured. \n         Each of these pages can be viewed in your browser. Add  .json  to the URL to get back the contents as JSON.", "sections_fts": 1, "rank": null}
{"rowid": 158, "title": "/-/metadata", "content": "Shows the contents of the  metadata.json  file that was passed to  datasette serve , if any.  Metadata example : \n             {\n    \"license\": \"CC Attribution 4.0 License\",\n    \"license_url\": \"http://creativecommons.org/licenses/by/4.0/\",\n    \"source\": \"fivethirtyeight/data on GitHub\",\n    \"source_url\": \"https://github.com/fivethirtyeight/data\",\n    \"title\": \"Five Thirty Eight\",\n    \"databases\": {\n\n    }\n}", "sections_fts": 1, "rank": null}
{"rowid": 159, "title": "/-/versions", "content": "Shows the version of Datasette, Python and SQLite.  Versions example : \n             {\n    \"datasette\": {\n        \"version\": \"0.60\"\n    },\n    \"python\": {\n        \"full\": \"3.8.12 (default, Dec 21 2021, 10:45:09) \\n[GCC 10.2.1 20210110]\",\n        \"version\": \"3.8.12\"\n    },\n    \"sqlite\": {\n        \"extensions\": {\n            \"json1\": null\n        },\n        \"fts_versions\": [\n            \"FTS5\",\n            \"FTS4\",\n            \"FTS3\"\n        ],\n        \"compile_options\": [\n            \"COMPILER=gcc-6.3.0 20170516\",\n            \"ENABLE_FTS3\",\n            \"ENABLE_FTS4\",\n            \"ENABLE_FTS5\",\n            \"ENABLE_JSON1\",\n            \"ENABLE_RTREE\",\n            \"THREADSAFE=1\"\n        ],\n        \"version\": \"3.37.0\"\n    }\n}", "sections_fts": 1, "rank": null}
{"rowid": 160, "title": "/-/plugins", "content": "Shows a list of currently installed plugins and their versions.  Plugins example : \n             [\n    {\n        \"name\": \"datasette_cluster_map\",\n        \"static\": true,\n        \"templates\": false,\n        \"version\": \"0.10\",\n        \"hooks\": [\"extra_css_urls\", \"extra_js_urls\", \"extra_body_script\"]\n    }\n] \n             Add  ?all=1  to include details of the default plugins baked into Datasette.", "sections_fts": 1, "rank": null}
{"rowid": 161, "title": "/-/settings", "content": "Shows the  Settings  for this instance of Datasette.  Settings example : \n             {\n    \"default_facet_size\": 30,\n    \"default_page_size\": 100,\n    \"facet_suggest_time_limit_ms\": 50,\n    \"facet_time_limit_ms\": 1000,\n    \"max_returned_rows\": 1000,\n    \"sql_time_limit_ms\": 1000\n}", "sections_fts": 1, "rank": null}
{"rowid": 162, "title": "/-/databases", "content": "Shows currently attached databases.  Databases example : \n             [\n    {\n        \"hash\": null,\n        \"is_memory\": false,\n        \"is_mutable\": true,\n        \"name\": \"fixtures\",\n        \"path\": \"fixtures.db\",\n        \"size\": 225280\n    }\n]", "sections_fts": 1, "rank": null}
{"rowid": 163, "title": "/-/threads", "content": "Shows details of threads and  asyncio  tasks.  Threads example : \n             {\n    \"num_threads\": 2,\n    \"threads\": [\n        {\n            \"daemon\": false,\n            \"ident\": 4759197120,\n            \"name\": \"MainThread\"\n        },\n        {\n            \"daemon\": true,\n            \"ident\": 123145319682048,\n            \"name\": \"Thread-1\"\n        },\n    ],\n    \"num_tasks\": 3,\n    \"tasks\": [\n        \"<Task pending coro=<RequestResponseCycle.run_asgi() running at uvicorn/protocols/http/httptools_impl.py:385> cb=[set.discard()]>\",\n        \"<Task pending coro=<Server.serve() running at uvicorn/main.py:361> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x10365c3d0>()]> cb=[run_until_complete.<locals>.<lambda>()]>\",\n        \"<Task pending coro=<LifespanOn.main() running at uvicorn/lifespan/on.py:48> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x10364f050>()]>>\"\n    ]\n}", "sections_fts": 1, "rank": null}
{"rowid": 164, "title": "/-/actor", "content": "Shows the currently authenticated actor. Useful for debugging Datasette authentication plugins. \n             {\n    \"actor\": {\n        \"id\": 1,\n        \"username\": \"some-user\"\n    }\n}", "sections_fts": 1, "rank": null}
{"rowid": 165, "title": "/-/messages", "content": "The debug tool at  /-/messages  can be used to set flash messages to try out that feature. See  .add_message(request, message, type=datasette.INFO)  for details of this feature.", "sections_fts": 1, "rank": null}
{"rowid": 166, "title": "Deploying Datasette", "content": "The quickest way to deploy a Datasette instance on the internet is to use the  datasette publish  command, described in  Publishing data . This can be used to quickly deploy Datasette to a number of hosting providers including Heroku, Google Cloud Run and Vercel. \n         You can deploy Datasette to other hosting providers using the instructions on this page.", "sections_fts": 1, "rank": null}
{"rowid": 167, "title": "Deployment fundamentals", "content": "Datasette can be deployed as a single  datasette  process that listens on a port. Datasette is not designed to be run as root, so that process should listen on a higher port such as port 8000. \n             If you want to serve Datasette on port 80 (the HTTP default port) or port 443 (for HTTPS) you should run it behind a proxy server, such as nginx, Apache or HAProxy. The proxy server can listen on port 80/443 and forward traffic on to Datasette.", "sections_fts": 1, "rank": null}
{"rowid": 168, "title": "Running Datasette using systemd", "content": "You can run Datasette on Ubuntu or Debian systems using  systemd . \n             First, ensure you have Python 3 and  pip  installed. On Ubuntu you can use  sudo apt-get install python3 python3-pip . \n             You can install Datasette into a virtual environment, or you can install it system-wide. To install system-wide, use  sudo pip3 install datasette . \n             Now create a folder for your Datasette databases, for example using  mkdir /home/ubuntu/datasette-root . \n             You can copy a test database into that folder like so: \n             cd /home/ubuntu/datasette-root\ncurl -O https://latest.datasette.io/fixtures.db \n             Create a file at  /etc/systemd/system/datasette.service  with the following contents: \n             [Unit]\nDescription=Datasette\nAfter=network.target\n\n[Service]\nType=simple\nUser=ubuntu\nEnvironment=DATASETTE_SECRET=\nWorkingDirectory=/home/ubuntu/datasette-root\nExecStart=datasette serve . -h 127.0.0.1 -p 8000\nRestart=on-failure\n\n[Install]\nWantedBy=multi-user.target \n             Add a random value for the  DATASETTE_SECRET  - this will be used to sign Datasette cookies such as the CSRF token cookie. You can generate a suitable value like so: \n             $ python3 -c 'import secrets; print(secrets.token_hex(32))' \n             This configuration will run Datasette against all database files contained in the  /home/ubuntu/datasette-root  directory. If that directory contains a  metadata.yml  (or  .json ) file or a  templates/  or  plugins/  sub-directory those will automatically be loaded by Datasette - see  Configuration directory mode  for details. \n             You can start the Datasette process running using the following: \n             sudo systemctl daemon-reload\nsudo systemctl start datasette.service \n             You will need to restart the Datasette service after making changes to its  metadata.json  configuration or adding a new database file to that directory. You can do that using: \n             sudo systemctl restart datasette.service \n             Once the service has started you can confirm that Datasette is running on port 8000 like so: \n             curl 127.0.0.1:8000/-/versions.json\n# Should output JSON showing the installed version \n             Datasette will not be accessible from outside the server because it is listening on  127.0.0.1 . You can expose it by instead listening on  0.0.0.0 , but a better way is to set up a proxy such as  nginx  - see  Running Datasette behind a proxy .", "sections_fts": 1, "rank": null}
{"rowid": 169, "title": "Running Datasette using OpenRC", "content": "OpenRC is the service manager on non-systemd Linux distributions like  Alpine Linux  and  Gentoo . \n             Create an init script at  /etc/init.d/datasette  with the following contents: \n             #!/sbin/openrc-run\n\nname=\"datasette\"\ncommand=\"datasette\"\ncommand_args=\"serve -h 0.0.0.0 /path/to/db.db\"\ncommand_background=true\npidfile=\"/run/${RC_SVCNAME}.pid\" \n             You then need to configure the service to run at boot and start it: \n             rc-update add datasette\nrc-service datasette start", "sections_fts": 1, "rank": null}
{"rowid": 170, "title": "Deploying using buildpacks", "content": "Some hosting providers such as  Heroku ,  DigitalOcean App Platform  and  Scalingo  support the  Buildpacks standard  for deploying Python web applications. \n             Deploying Datasette on these platforms requires two files:  requirements.txt  and  Procfile . \n             The  requirements.txt  file lets the platform know which Python packages should be installed. It should contain  datasette  at a minimum, but can also list any Datasette plugins you wish to install - for example: \n             datasette\ndatasette-vega \n             The  Procfile  lets the hosting platform know how to run the command that serves web traffic. It should look like this: \n             web: datasette . -h 0.0.0.0 -p $PORT --cors \n             The  $PORT  environment variable is provided by the hosting platform.  --cors  enables CORS requests from JavaScript running on other websites to your domain - omit this if you don't want to allow CORS. You can add additional Datasette  Settings  options here too. \n             These two files should be enough to deploy Datasette on any host that supports buildpacks. Datasette will serve any SQLite files that are included in the root directory of the application. \n             If you want to build SQLite files or download them as part of the deployment process you can do so using a  bin/post_compile  file. For example, the following  bin/post_compile  will download an example database that will then be served by Datasette: \n             wget https://fivethirtyeight.datasettes.com/fivethirtyeight.db \n             simonw/buildpack-datasette-demo  is an example GitHub repository showing a Datasette configuration that can be deployed to a buildpack-supporting host.", "sections_fts": 1, "rank": null}
{"rowid": 171, "title": "Running Datasette behind a proxy", "content": "You may wish to run Datasette behind an Apache or nginx proxy, using a path within your existing site. \n             You can use the  base_url  configuration setting to tell Datasette to serve traffic with a specific URL prefix. For example, you could run Datasette like this: \n             datasette my-database.db --setting base_url /my-datasette/ -p 8009 \n             This will run Datasette with the following URLs: \n             \n                 \n                     http://127.0.0.1:8009/my-datasette/  - the Datasette homepage \n                 \n                 \n                     http://127.0.0.1:8009/my-datasette/my-database  - the page for the  my-database.db  database \n                 \n                 \n                     http://127.0.0.1:8009/my-datasette/my-database/some_table  - the page for the  some_table  table \n                 \n             \n             You can now set your nginx or Apache server to proxy the  /my-datasette/  path to this Datasette instance.", "sections_fts": 1, "rank": null}
{"rowid": 172, "title": "Nginx proxy configuration", "content": "Here is an example of an  nginx  configuration file that will proxy traffic to Datasette: \n                 daemon off;\n\nevents {\n  worker_connections  1024;\n}\nhttp {\n  server {\n    listen 80;\n    location /my-datasette {\n      proxy_pass http://127.0.0.1:8009/my-datasette;\n      proxy_set_header Host $host;\n    }\n  }\n} \n                 You can also use the  --uds  option to Datasette to listen on a Unix domain socket instead of a port, configuring the nginx upstream proxy like this: \n                 daemon off;\nevents {\n  worker_connections  1024;\n}\nhttp {\n  server {\n    listen 80;\n    location /my-datasette {\n      proxy_pass http://datasette/my-datasette;\n      proxy_set_header Host $host;\n    }\n  }\n  upstream datasette {\n    server unix:/tmp/datasette.sock;\n  }\n} \n                 Then run Datasette with  datasette --uds /tmp/datasette.sock path/to/database.db --setting base_url /my-datasette/ .", "sections_fts": 1, "rank": null}
{"rowid": 173, "title": "Apache proxy configuration", "content": "For  Apache , you can use the  ProxyPass  directive. First make sure the following lines are uncommented: \n                 LoadModule proxy_module lib/httpd/modules/mod_proxy.so\nLoadModule proxy_http_module lib/httpd/modules/mod_proxy_http.so \n                 Then add these directives to proxy traffic: \n                 ProxyPass /my-datasette/ http://127.0.0.1:8009/my-datasette/\nProxyPreserveHost On \n                 A live demo of Datasette running behind Apache using this proxy setup can be seen at  datasette-apache-proxy-demo.datasette.io/prefix/ . The code for that demo can be found in the  demos/apache-proxy  directory. \n                 Using  --uds  you can use Unix domain sockets similar to the nginx example: \n                 ProxyPass /my-datasette/ unix:/tmp/datasette.sock|http://localhost/my-datasette/ \n                 The  ProxyPreserveHost On  directive ensures that the original  Host:  header from the incoming request is passed through to Datasette. Datasette needs this to correctly assemble links to other pages using the  .absolute_url(request, path)  method.", "sections_fts": 1, "rank": null}
{"rowid": 174, "title": "Plugins", "content": "Datasette's plugin system allows additional features to be implemented as Python\n            code (or front-end JavaScript) which can be wrapped up in a separate Python\n            package. The underlying mechanism uses  pluggy . \n         See the  Datasette plugins directory  for a list of existing plugins, or take a look at the\n             datasette-plugin  topic on GitHub. \n         Things you can do with plugins include: \n         \n             \n                 Add visualizations to Datasette, for example\n                     datasette-cluster-map  and\n                     datasette-vega . \n             \n             \n                 Make new custom SQL functions available for use within Datasette, for example\n                     datasette-haversine  and\n                     datasette-jellyfish . \n             \n             \n                 Define custom output formats with custom extensions, for example  datasette-atom  and\n                     datasette-ics . \n             \n             \n                 Add template functions that can be called within your Jinja custom templates,\n                    for example  datasette-render-markdown . \n             \n             \n                 Customize how database values are rendered in the Datasette interface, for example\n                     datasette-render-binary  and\n                     datasette-pretty-json . \n             \n             \n                 Customize how Datasette's authentication and permissions systems work, for example  datasette-auth-tokens  and\n                     datasette-permissions-sql .", "sections_fts": 1, "rank": null}
{"rowid": 175, "title": "Installing plugins", "content": "If a plugin has been packaged for distribution using setuptools you can use the plugin by installing it alongside Datasette in the same virtual environment or Docker container. \n             You can install plugins using the  datasette install  command: \n             datasette install datasette-vega \n             You can uninstall plugins with  datasette uninstall : \n             datasette uninstall datasette-vega \n             You can upgrade plugins with  datasette install --upgrade  or  datasette install -U : \n             datasette install -U datasette-vega \n             This command can also be used to upgrade Datasette itself to the latest released version: \n             datasette install -U datasette \n             These commands are thin wrappers around  pip install  and  pip uninstall , which ensure they run  pip  in the same virtual environment as Datasette itself.", "sections_fts": 1, "rank": null}
{"rowid": 176, "title": "One-off plugins using --plugins-dir", "content": "You can also define one-off per-project plugins by saving them as  plugin_name.py  functions in a  plugins/  folder and then passing that folder to  datasette  using the  --plugins-dir  option: \n                 datasette mydb.db --plugins-dir=plugins/", "sections_fts": 1, "rank": null}
{"rowid": 177, "title": "Deploying plugins using datasette publish", "content": "The  datasette publish  and  datasette package  commands both take an optional  --install  argument. You can use this one or more times to tell Datasette to  pip install  specific plugins as part of the process: \n                 datasette publish cloudrun mydb.db --install=datasette-vega \n                 You can use the name of a package on PyPI or any of the other valid arguments to  pip install  such as a URL to a  .zip  file: \n                 datasette publish cloudrun mydb.db \\\n    --install=https://url-to-my-package.zip", "sections_fts": 1, "rank": null}
{"rowid": 178, "title": "Seeing what plugins are installed", "content": "You can see a list of installed plugins by navigating to the  /-/plugins  page of your Datasette instance - for example:  https://fivethirtyeight.datasettes.com/-/plugins \n             You can also use the  datasette plugins  command: \n             $ datasette plugins\n[\n    {\n        \"name\": \"datasette_json_html\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": \"0.4.0\"\n    }\n] \n             [[[cog\nfrom datasette import cli\nfrom click.testing import CliRunner\nimport textwrap, json\ncog.out(\"\\n\")\nresult = CliRunner().invoke(cli.cli, [\"plugins\", \"--all\"])\n# cog.out() with text containing newlines was unindenting for some reason\ncog.outl(\"If you run ``datasette plugins --all`` it will include default plugins that ship as part of Datasette::\\n\")\nplugins = [p for p in json.loads(result.output) if p[\"name\"].startswith(\"datasette.\")]\nindented = textwrap.indent(json.dumps(plugins, indent=4), \"    \")\nfor line in indented.split(\"\\n\"):\n    cog.outl(line)\ncog.out(\"\\n\\n\") \n             ]]] \n             If you run  datasette plugins --all  it will include default plugins that ship as part of Datasette: \n             [\n    {\n        \"name\": \"datasette.actor_auth_cookie\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"actor_from_request\"\n        ]\n    },\n    {\n        \"name\": \"datasette.blob_renderer\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"register_output_renderer\"\n        ]\n    },\n    {\n        \"name\": \"datasette.default_magic_parameters\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"register_magic_parameters\"\n        ]\n    },\n    {\n        \"name\": \"datasette.default_menu_links\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"menu_links\"\n        ]\n    },\n    {\n        \"name\": \"datasette.default_permissions\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"permission_allowed\"\n        ]\n    },\n    {\n        \"name\": \"datasette.facets\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"register_facet_classes\"\n        ]\n    },\n    {\n        \"name\": \"datasette.filters\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"filters_from_request\"\n        ]\n    },\n    {\n        \"name\": \"datasette.forbidden\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"forbidden\"\n        ]\n    },\n    {\n        \"name\": \"datasette.handle_exception\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"handle_exception\"\n        ]\n    },\n    {\n        \"name\": \"datasette.publish.cloudrun\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"publish_subcommand\"\n        ]\n    },\n    {\n        \"name\": \"datasette.publish.heroku\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"publish_subcommand\"\n        ]\n    },\n    {\n        \"name\": \"datasette.sql_functions\",\n        \"static\": false,\n        \"templates\": false,\n        \"version\": null,\n        \"hooks\": [\n            \"prepare_connection\"\n        ]\n    }\n] \n             [[[end]]] \n             You can add the  --plugins-dir=  option to include any plugins found in that directory.", "sections_fts": 1, "rank": null}
{"rowid": 179, "title": "Plugin configuration", "content": "Plugins can have their own configuration, embedded in a  Metadata  file. Configuration options for plugins live within a  \"plugins\"  key in that file, which can be included at the root, database or table level. \n             Here is an example of some plugin configuration for a specific table: \n             {\n    \"databases\": {\n        \"sf-trees\": {\n            \"tables\": {\n                \"Street_Tree_List\": {\n                    \"plugins\": {\n                        \"datasette-cluster-map\": {\n                            \"latitude_column\": \"lat\",\n                            \"longitude_column\": \"lng\"\n                        }\n                    }\n                }\n            }\n        }\n    }\n} \n             This tells the  datasette-cluster-map  column which latitude and longitude columns should be used for a table called  Street_Tree_List  inside a database file called  sf-trees.db .", "sections_fts": 1, "rank": null}
{"rowid": 180, "title": "Secret configuration values", "content": "Any values embedded in  metadata.json  will be visible to anyone who views the  /-/metadata  page of your Datasette instance. Some plugins may need configuration that should stay secret - API keys for example. There are two ways in which you can store secret configuration values. \n                 As environment variables . If your secret lives in an environment variable that is available to the Datasette process, you can indicate that the configuration value should be read from that environment variable like so: \n                 {\n    \"plugins\": {\n        \"datasette-auth-github\": {\n            \"client_secret\": {\n                \"$env\": \"GITHUB_CLIENT_SECRET\"\n            }\n        }\n    }\n} \n                 As values in separate files . Your secrets can also live in files on disk. To specify a secret should be read from a file, provide the full file path like this: \n                 {\n    \"plugins\": {\n        \"datasette-auth-github\": {\n            \"client_secret\": {\n                \"$file\": \"/secrets/client-secret\"\n            }\n        }\n    }\n} \n                 If you are publishing your data using the  datasette publish  family of commands, you can use the  --plugin-secret  option to set these secrets at publish time. For example, using Heroku you might run the following command: \n                 $ datasette publish heroku my_database.db \\\n    --name my-heroku-app-demo \\\n    --install=datasette-auth-github \\\n    --plugin-secret datasette-auth-github client_id your_client_id \\\n    --plugin-secret datasette-auth-github client_secret your_client_secret \n                 This will set the necessary environment variables and add the following to the deployed  metadata.json : \n                 {\n    \"plugins\": {\n        \"datasette-auth-github\": {\n            \"client_id\": {\n                \"$env\": \"DATASETTE_AUTH_GITHUB_CLIENT_ID\"\n            },\n            \"client_secret\": {\n                \"$env\": \"DATASETTE_AUTH_GITHUB_CLIENT_SECRET\"\n            }\n        }\n    }\n}", "sections_fts": 1, "rank": null}
{"rowid": 181, "title": "Internals for plugins", "content": "Many  Plugin hooks  are passed objects that provide access to internal Datasette functionality. The interface to these objects should not be considered stable with the exception of methods that are documented here.", "sections_fts": 1, "rank": null}
{"rowid": 182, "title": "Request object", "content": "The request object is passed to various plugin hooks. It represents an incoming HTTP request. It has the following properties: \n             \n                 \n                     .scope  - dictionary \n                     \n                         The ASGI scope that was used to construct this request, described in the  ASGI HTTP connection scope  specification. \n                     \n                 \n                 \n                     .method  - string \n                     \n                         The HTTP method for this request, usually  GET  or  POST . \n                     \n                 \n                 \n                     .url  - string \n                     \n                         The full URL for this request, e.g.  https://latest.datasette.io/fixtures . \n                     \n                 \n                 \n                     .scheme  - string \n                     \n                         The request scheme - usually  https  or  http . \n                     \n                 \n                 \n                     .headers  - dictionary (str -> str) \n                     \n                         A dictionary of incoming HTTP request headers. Header names have been converted to lowercase. \n                     \n                 \n                 \n                     .cookies  - dictionary (str -> str) \n                     \n                         A dictionary of incoming cookies \n                     \n                 \n                 \n                     .host  - string \n                     \n                         The host header from the incoming request, e.g.  latest.datasette.io  or  localhost . \n                     \n                 \n                 \n                     .path  - string \n                     \n                         The path of the request excluding the query string, e.g.  /fixtures . \n                     \n                 \n                 \n                     .full_path  - string \n                     \n                         The path of the request including the query string if one is present, e.g.  /fixtures?sql=select+sqlite_version() . \n                     \n                 \n                 \n                     .query_string  - string \n                     \n                         The query string component of the request, without the  ?  - e.g.  name__contains=sam&age__gt=10 . \n                     \n                 \n                 \n                     .args  - MultiParams \n                     \n                         An object representing the parsed query string parameters, see below. \n                     \n                 \n                 \n                     .url_vars  - dictionary (str -> str) \n                     \n                         Variables extracted from the URL path, if that path was defined using a regular expression. See  register_routes(datasette) . \n                     \n                 \n                 \n                     .actor  - dictionary (str -> Any) or None \n                     \n                         The currently authenticated actor (see  actors ), or  None  if the request is unauthenticated. \n                     \n                 \n             \n             The object also has two awaitable methods: \n             \n                 \n                     await request.post_vars()  - dictionary \n                     \n                         Returns a dictionary of form variables that were submitted in the request body via  POST . Don't forget to read about  CSRF protection ! \n                     \n                 \n                 \n                     await request.post_body()  - bytes \n                     \n                         Returns the un-parsed body of a request submitted by  POST  - useful for things like incoming JSON data. \n                     \n                 \n             \n             And a class method that can be used to create fake request objects for use in tests: \n             \n                 \n                     fake(path_with_query_string, method=\"GET\", scheme=\"http\", url_vars=None) \n                     \n                         Returns a  Request  instance for the specified path and method. For example: \n                         from datasette import Request\nfrom pprint import pprint\n\nrequest = Request.fake(\n    \"/fixtures/facetable/\",\n    url_vars={\"database\": \"fixtures\", \"table\": \"facetable\"},\n)\npprint(request.scope) \n                         This outputs: \n                         {'http_version': '1.1',\n 'method': 'GET',\n 'path': '/fixtures/facetable/',\n 'query_string': b'',\n 'raw_path': b'/fixtures/facetable/',\n 'scheme': 'http',\n 'type': 'http',\n 'url_route': {'kwargs': {'database': 'fixtures', 'table': 'facetable'}}}", "sections_fts": 1, "rank": null}
{"rowid": 183, "title": "The MultiParams class", "content": "request.args  is a  MultiParams  object - a dictionary-like object which provides access to query string parameters that may have multiple values. \n             Consider the query string  ?foo=1&foo=2&bar=3  - with two values for  foo  and one value for  bar . \n             \n                 \n                     request.args[key]  - string \n                     \n                         Returns the first value for that key, or raises a  KeyError  if the key is missing. For the above example  request.args[\"foo\"]  would return  \"1\" . \n                     \n                 \n                 \n                     request.args.get(key)  - string or None \n                     \n                         Returns the first value for that key, or  None  if the key is missing. Pass a second argument to specify a different default, e.g.  q = request.args.get(\"q\", \"\") . \n                     \n                 \n                 \n                     request.args.getlist(key)  - list of strings \n                     \n                         Returns the list of strings for that key.  request.args.getlist(\"foo\")  would return  [\"1\", \"2\"]  in the above example.  request.args.getlist(\"bar\")  would return  [\"3\"] . If the key is missing an empty list will be returned. \n                     \n                 \n                 \n                     request.args.keys()  - list of strings \n                     \n                         Returns the list of available keys - for the example this would be  [\"foo\", \"bar\"] . \n                     \n                 \n                 \n                     key in request.args  - True or False \n                     \n                         You can use  if key in request.args  to check if a key is present. \n                     \n                 \n                 \n                     for key in request.args  - iterator \n                     \n                         This lets you loop through every available key. \n                     \n                 \n                 \n                     len(request.args)  - integer \n                     \n                         Returns the number of keys.", "sections_fts": 1, "rank": null}
{"rowid": 184, "title": "Response class", "content": "The  Response  class can be returned from view functions that have been registered using the  register_routes(datasette)  hook. \n             The  Response()  constructor takes the following arguments: \n             \n                 \n                     body  - string \n                     \n                         The body of the response. \n                     \n                 \n                 \n                     status  - integer (optional) \n                     \n                         The HTTP status - defaults to 200. \n                     \n                 \n                 \n                     headers  - dictionary (optional) \n                     \n                         A dictionary of extra HTTP headers, e.g.  {\"x-hello\": \"world\"} . \n                     \n                 \n                 \n                     content_type  - string (optional) \n                     \n                         The content-type for the response. Defaults to  text/plain . \n                     \n                 \n             \n             For example: \n             from datasette.utils.asgi import Response\n\nresponse = Response(\n    \"<xml>This is XML</xml>\",\n    content_type=\"application/xml; charset=utf-8\",\n) \n             The quickest way to create responses is using the  Response.text(...) ,  Response.html(...) ,  Response.json(...)  or  Response.redirect(...)  helper methods: \n             from datasette.utils.asgi import Response\n\nhtml_response = Response.html(\"This is HTML\")\njson_response = Response.json({\"this_is\": \"json\"})\ntext_response = Response.text(\n    \"This will become utf-8 encoded text\"\n)\n# Redirects are served as 302, unless you pass status=301:\nredirect_response = Response.redirect(\n    \"https://latest.datasette.io/\"\n) \n             Each of these responses will use the correct corresponding content-type -  text/html; charset=utf-8 ,  application/json; charset=utf-8  or  text/plain; charset=utf-8  respectively. \n             Each of the helper methods take optional  status=  and  headers=  arguments, documented above.", "sections_fts": 1, "rank": null}
{"rowid": 185, "title": "Returning a response with .asgi_send(send)", "content": "In most cases you will return  Response  objects from your own view functions. You can also use a  Response  instance to respond at a lower level via ASGI, for example if you are writing code that uses the  asgi_wrapper(datasette)  hook. \n                 Create a  Response  object and then use  await response.asgi_send(send) , passing the ASGI  send  function. For example: \n                 async def require_authorization(scope, receive, send):\n    response = Response.text(\n        \"401 Authorization Required\",\n        headers={\n            \"www-authenticate\": 'Basic realm=\"Datasette\", charset=\"UTF-8\"'\n        },\n        status=401,\n    )\n    await response.asgi_send(send)", "sections_fts": 1, "rank": null}
{"rowid": 186, "title": "Setting cookies with response.set_cookie()", "content": "To set cookies on the response, use the  response.set_cookie(...)  method. The method signature looks like this: \n                 def set_cookie(\n    self,\n    key,\n    value=\"\",\n    max_age=None,\n    expires=None,\n    path=\"/\",\n    domain=None,\n    secure=False,\n    httponly=False,\n    samesite=\"lax\",\n): ... \n                 You can use this with  datasette.sign()  to set signed cookies. Here's how you would set the  ds_actor cookie  for use with Datasette  authentication : \n                 response = Response.redirect(\"/\")\nresponse.set_cookie(\n    \"ds_actor\",\n    datasette.sign({\"a\": {\"id\": \"cleopaws\"}}, \"actor\"),\n)\nreturn response", "sections_fts": 1, "rank": null}
{"rowid": 187, "title": "Datasette class", "content": "This object is an instance of the  Datasette  class, passed to many plugin hooks as an argument called  datasette . \n             You can create your own instance of this - for example to help write tests for a plugin - like so: \n             from datasette.app import Datasette\n\n# With no arguments a single in-memory database will be attached\ndatasette = Datasette()\n\n# The files= argument can load files from disk\ndatasette = Datasette(files=[\"/path/to/my-database.db\"])\n\n# Pass metadata as a JSON dictionary like this\ndatasette = Datasette(\n    files=[\"/path/to/my-database.db\"],\n    metadata={\n        \"databases\": {\n            \"my-database\": {\n                \"description\": \"This is my database\"\n            }\n        }\n    },\n) \n             Constructor parameters include: \n             \n                 \n                     files=[...]  - a list of database files to open \n                 \n                 \n                     immutables=[...]  - a list of database files to open in immutable mode \n                 \n                 \n                     metadata={...}  - a dictionary of  Metadata \n                 \n                 \n                     config_dir=...  - the  configuration directory  to use, stored in  datasette.config_dir", "sections_fts": 1, "rank": null}
{"rowid": 188, "title": ".databases", "content": "Property exposing a  collections.OrderedDict  of databases currently connected to Datasette. \n                 The dictionary keys are the name of the database that is used in the URL - e.g.  /fixtures  would have a key of  \"fixtures\" . The values are  Database class  instances. \n                 All databases are listed, irrespective of user permissions. This means that the  _internal  database will always be listed here.", "sections_fts": 1, "rank": null}
{"rowid": 189, "title": ".plugin_config(plugin_name, database=None, table=None)", "content": "plugin_name  - string \n                         \n                             The name of the plugin to look up configuration for. Usually this is something similar to  datasette-cluster-map . \n                         \n                     \n                     \n                         database  - None or string \n                         \n                             The database the user is interacting with. \n                         \n                     \n                     \n                         table  - None or string \n                         \n                             The table the user is interacting with. \n                         \n                     \n                 \n                 This method lets you read plugin configuration values that were set in  metadata.json . See  Writing plugins that accept configuration  for full details of how this method should be used. \n                 The return value will be the value from the configuration file - usually a dictionary. \n                 If the plugin is not configured the return value will be  None .", "sections_fts": 1, "rank": null}
{"rowid": 190, "title": "await .render_template(template, context=None, request=None)", "content": "template  - string, list of strings or jinja2.Template \n                         \n                             The template file to be rendered, e.g.  my_plugin.html . Datasette will search for this file first in the  --template-dir=  location, if it was specified - then in the plugin's bundled templates and finally in Datasette's set of default templates. \n                             If this is a list of template file names then the first one that exists will be loaded and rendered. \n                             If this is a Jinja  Template object  it will be used directly. \n                         \n                     \n                     \n                         context  - None or a Python dictionary \n                         \n                             The context variables to pass to the template. \n                         \n                     \n                     \n                         request  - request object or None \n                         \n                             If you pass a Datasette request object here it will be made available to the template. \n                         \n                     \n                 \n                 Renders a  Jinja template  using Datasette's preconfigured instance of Jinja and returns the resulting string. The template will have access to Datasette's default template functions and any functions that have been made available by other plugins.", "sections_fts": 1, "rank": null}
{"rowid": 191, "title": "await .permission_allowed(actor, action, resource=None, default=False)", "content": "actor  - dictionary \n                         \n                             The authenticated actor. This is usually  request.actor . \n                         \n                     \n                     \n                         action  - string \n                         \n                             The name of the action that is being permission checked. \n                         \n                     \n                     \n                         resource  - string or tuple, optional \n                         \n                             The resource, e.g. the name of the database, or a tuple of two strings containing the name of the database and the name of the table. Only some permissions apply to a resource. \n                         \n                     \n                     \n                         default  - optional, True or False \n                         \n                             Should this permission check be default allow or default deny. \n                         \n                     \n                 \n                 Check if the given actor has  permission  to perform the given action on the given resource. \n                 Some permission checks are carried out against  rules defined in metadata.json , while other custom permissions may be decided by plugins that implement the  permission_allowed(datasette, actor, action, resource)  plugin hook. \n                 If neither  metadata.json  nor any of the plugins provide an answer to the permission query the  default  argument will be returned. \n                 See  Built-in permissions  for a full list of permission actions included in Datasette core.", "sections_fts": 1, "rank": null}
{"rowid": 192, "title": "await .ensure_permissions(actor, permissions)", "content": "actor  - dictionary \n                         \n                             The authenticated actor. This is usually  request.actor . \n                         \n                     \n                     \n                         permissions  - list \n                         \n                             A list of permissions to check. Each permission in that list can be a string  action  name or a 2-tuple of  (action, resource) . \n                         \n                     \n                 \n                 This method allows multiple permissions to be checked at once. It raises a  datasette.Forbidden  exception if any of the checks are denied before one of them is explicitly granted. \n                 This is useful when you need to check multiple permissions at once. For example, an actor should be able to view a table if either one of the following checks returns  True  or not a single one of them returns  False : \n                 await self.ds.ensure_permissions(\n    request.actor,\n    [\n        (\"view-table\", (database, table)),\n        (\"view-database\", database),\n        \"view-instance\",\n    ],\n)", "sections_fts": 1, "rank": null}
{"rowid": 193, "title": "await .check_visibility(actor, action=None, resource=None, permissions=None)", "content": "actor  - dictionary \n                         \n                             The authenticated actor. This is usually  request.actor . \n                         \n                     \n                     \n                         action  - string, optional \n                         \n                             The name of the action that is being permission checked. \n                         \n                     \n                     \n                         resource  - string or tuple, optional \n                         \n                             The resource, e.g. the name of the database, or a tuple of two strings containing the name of the database and the name of the table. Only some permissions apply to a resource. \n                         \n                     \n                     \n                         permissions  - list of  action  strings or  (action, resource)  tuples, optional \n                         \n                             Provide this instead of  action  and  resource  to check multiple permissions at once. \n                         \n                     \n                 \n                 This convenience method can be used to answer the question \"should this item be considered private, in that it is visible to me but it is not visible to anonymous users?\" \n                 It returns a tuple of two booleans,  (visible, private) .  visible  indicates if the actor can see this resource.  private  will be  True  if an anonymous user would not be able to view the resource. \n                 This example checks if the user can access a specific table, and sets  private  so that a padlock icon can later be displayed: \n                 visible, private = await self.ds.check_visibility(\n    request.actor,\n    action=\"view-table\",\n    resource=(database, table),\n) \n                 The following example runs three checks in a row, similar to  await .ensure_permissions(actor, permissions) . If any of the checks are denied before one of them is explicitly granted then  visible  will be  False .  private  will be  True  if an anonymous user would not be able to view the resource. \n                 visible, private = await self.ds.check_visibility(\n    request.actor,\n    permissions=[\n        (\"view-table\", (database, table)),\n        (\"view-database\", database),\n        \"view-instance\",\n    ],\n)", "sections_fts": 1, "rank": null}
{"rowid": 194, "title": ".get_database(name)", "content": "name  - string, optional \n                         \n                             The name of the database - optional. \n                         \n                     \n                 \n                 Returns the specified database object. Raises a  KeyError  if the database does not exist. Call this method without an argument to return the first connected database.", "sections_fts": 1, "rank": null}
{"rowid": 195, "title": ".add_database(db, name=None, route=None)", "content": "db  - datasette.database.Database instance \n                         \n                             The database to be attached. \n                         \n                     \n                     \n                         name  - string, optional \n                         \n                             The name to be used for this database . If not specified Datasette will pick one based on the filename or memory name. \n                         \n                     \n                     \n                         route  - string, optional \n                         \n                             This will be used in the URL path. If not specified, it will default to the same thing as the  name . \n                         \n                     \n                 \n                 The  datasette.add_database(db)  method lets you add a new database to the current Datasette instance. \n                 The  db  parameter should be an instance of the  datasette.database.Database  class. For example: \n                 from datasette.database import Database\n\ndatasette.add_database(\n    Database(\n        datasette,\n        path=\"path/to/my-new-database.db\",\n    )\n) \n                 This will add a mutable database and serve it at  /my-new-database . \n                 Use  is_mutable=False  to add an immutable database. \n                 .add_database()  returns the Database instance, with its name set as the  database.name  attribute. Any time you are working with a newly added database you should use the return value of  .add_database() , for example: \n                 db = datasette.add_database(\n    Database(datasette, memory_name=\"statistics\")\n)\nawait db.execute_write(\n    \"CREATE TABLE foo(id integer primary key)\"\n)", "sections_fts": 1, "rank": null}
{"rowid": 196, "title": ".add_memory_database(name)", "content": "Adds a shared in-memory database with the specified name: \n                 datasette.add_memory_database(\"statistics\") \n                 This is a shortcut for the following: \n                 from datasette.database import Database\n\ndatasette.add_database(\n    Database(datasette, memory_name=\"statistics\")\n) \n                 Using either of these pattern will result in the in-memory database being served at  /statistics .", "sections_fts": 1, "rank": null}
{"rowid": 197, "title": ".remove_database(name)", "content": "name  - string \n                         \n                             The name of the database to be removed. \n                         \n                     \n                 \n                 This removes a database that has been previously added.  name=  is the unique name of that database.", "sections_fts": 1, "rank": null}
{"rowid": 198, "title": ".sign(value, namespace=\"default\")", "content": "value  - any serializable type \n                         \n                             The value to be signed. \n                         \n                     \n                     \n                         namespace  - string, optional \n                         \n                             An alternative namespace, see the  itsdangerous salt documentation . \n                         \n                     \n                 \n                 Utility method for signing values, such that you can safely pass data to and from an untrusted environment. This is a wrapper around the  itsdangerous  library. \n                 This method returns a signed string, which can be decoded and verified using  .unsign(value, namespace=\"default\") .", "sections_fts": 1, "rank": null}
{"rowid": 199, "title": ".unsign(value, namespace=\"default\")", "content": "signed  - any serializable type \n                         \n                             The signed string that was created using  .sign(value, namespace=\"default\") . \n                         \n                     \n                     \n                         namespace  - string, optional \n                         \n                             The alternative namespace, if one was used. \n                         \n                     \n                 \n                 Returns the original, decoded object that was passed to  .sign(value, namespace=\"default\") . If the signature is not valid this raises a  itsdangerous.BadSignature  exception.", "sections_fts": 1, "rank": null}
{"rowid": 200, "title": ".add_message(request, message, type=datasette.INFO)", "content": "request  - Request \n                         \n                             The current Request object \n                         \n                     \n                     \n                         message  - string \n                         \n                             The message string \n                         \n                     \n                     \n                         type  - constant, optional \n                         \n                             The message type -  datasette.INFO ,  datasette.WARNING  or  datasette.ERROR \n                         \n                     \n                 \n                 Datasette's flash messaging mechanism allows you to add a message that will be displayed to the user on the next page that they visit. Messages are persisted in a  ds_messages  cookie. This method adds a message to that cookie. \n                 You can try out these messages (including the different visual styling of the three message types) using the  /-/messages  debugging tool.", "sections_fts": 1, "rank": null}