Reports · fetched 2026-09-30 · source: CISA .gov registry
The government web census, September 2026: what 16,783 .gov domains publish for people, security researchers and AI agents
Every registered .gov domain in CISA's public list (16,783 domains: federal, state, county, city, tribal, school and special districts) was fetched once over HTTPS on 2026-09-30, with robots.txt, security.txt (both locations), llms.txt and the A2A agent card. 13,060 answered with a website; the rest are registered for email or internal use, are not yet live, or blocked our one polite request.
What we found
- security.txt: 343 of 13,060 reachable sites (2.6%) publish one; 243 are complete and current (Contact and an unexpired Expires line, RFC 9116); 63 have expired. Federal executive-branch domains, which CISA's Binding Operational Directive 20-01 requires to publish a vulnerability disclosure policy and security.txt: 74 of 992 reachable (7.5%).
- Accessibility: 39.4% of reachable sites link an accessibility statement from the homepage (state: 38.2%, county: 40.7%, city: 39.1%). The ADA Title II web rule applies to public entities with 50,000 or more people from April 24, 2026 and to smaller ones from April 26, 2027.
- Privacy: 34.5% link a privacy policy from the homepage; 8.1% link FOIA or public-records information.
- HTTPS: 739 registered domains answer only over plain HTTP; 36.3% of reachable sites send HSTS; 679 .gov homepages redirect to a non-.gov host.
- AI agents: 9,777 sites serve a robots.txt; 524 block GPTBot, 504 ClaudeBot and 396 ChatGPT-User; 888 name an AI crawler explicitly; 126 carry a Content-Signal line. 718 publish an llms.txt and 1 an A2A agent card.
By level of government
| Level | Registered | Reachable | security.txt | valid | privacy link | accessibility link | HSTS | blocks GPTBot | llms.txt |
|---|---|---|---|---|---|---|---|---|---|
| Federal | 1,317 | 992 (75.3%) | 74 (7.5%) | 41 | 55.5% | 48.5% | 88.2% | 43 | 18 |
| State or territory | 1,483 | 992 (66.9%) | 58 (5.8%) | 20 | 38.0% | 38.2% | 44.4% | 43 | 34 |
| County | 3,016 | 2,291 (76.0%) | 25 (1.1%) | 13 | 30.2% | 40.7% | 30.4% | 105 | 127 |
| City | 9,467 | 7,667 (81.0%) | 166 (2.2%) | 153 | 32.2% | 39.1% | 28.9% | 314 | 436 |
| Tribal | 293 | 191 (65.2%) | 2 (1.0%) | 1 | 18.3% | 4.2% | 37.2% | 4 | 14 |
| School district | 74 | 43 (58.1%) | 5 (11.6%) | 5 | 34.9% | 51.2% | 55.8% | 2 | 1 |
| Special district | 1,102 | 858 (77.9%) | 13 (1.5%) | 10 | 41.4% | 36.5% | 47.4% | 13 | 88 |
Federal domains without a security.txt (first 60 of 400)
acus.gov · arctic.gov · goldwaterinternship.gov · afrh.gov · arc.gov · ffiec.gov · access-board.gov · abmc.gov · fbiic.gov · whistleblower.gov · billetes.gov · americorps.gov · joinamericorps.gov · nationalservice.gov · mlkday.gov · goldwaterscholarship.gov · fedpartnership.gov · federalreserve.gov · cia.gov · smartcheck.gov · cftc.gov · asc.gov · mentor.gov · serve.gov · dnfsb.gov · aicenter.gov · presidentialserviceawards.gov · anchorit.gov · recalls.gov · aiexports.gov · saferproducts.gov · eda.gov · aviationweather.gov · pandemicoversight.gov · heat.gov · oasis.gov · dietaryguidelines.gov · nafri.gov · ars-grin.gov · usda.gov · farmerfairness.gov · symbols.gov · myplate.gov · safetyvideos.gov · bea.gov · mbda.gov · climate.gov · csb.gov · nwcg.gov · coldcaserecords.gov · uscurrency.gov · saferproduct.gov · firstnet.gov · lcacommons.gov · rural.gov · federalreserveconsumerhelp.gov · mgi.gov · xd.gov · poolsafely.gov · edison.gov
Government sites that tell ChatGPT and Claude not to open pages for users
aftonmn.gov · alton-tx.gov · appletonmn.gov · booneiowa.gov · atwatermn.gov · baraboowi.gov · belgradecitymn.gov · averytx.gov · belleplaineiowa.gov · beloitwi.gov · blackbrookwi.gov · bolingbrook.gov · brahammn.gov · bethlehemnj.gov · bradentonfl.gov · brockwaytwpmn.gov · braselton.gov · brunswickmd.gov · brysoncitync.gov · camarillo.gov · byesvilleoh.gov · canyonlakeca.gov · cedarpointnc.gov · charlestownri.gov · chattahoocheefl.gov · centralcityia.gov · cedartowngeorgia.gov · chathamnewyork.gov · chiltonwi.gov · circlepinesmn.gov · cityofbensonmn.gov · cityofbyronmn.gov · cityofclaremontmn.gov · cityofclearlakemn.gov · cityofcorningny.gov · cityofclevelandmn.gov · cityofguttenbergia.gov · cityofkenyon.gov · cityoflagrangemo.gov · cityofmarionwi.gov
Government sites publishing an llms.txt
adriantownshipwi.gov · adriantwpmi.gov · alberttwpmi.gov · almacenterwi.gov · abitaspringsla.gov · amalgautah.gov · altontownshipil.gov · alvo.gov · argonnewi.gov · barringtontwpil.gov · annettatx.gov · aspinwallpa.gov · ashtonid.gov · arroyopr.gov · bearriverwy.gov · beavertwp-oh.gov · belgiumwi.gov · auburnmi.gov · beechgrove.gov · assyriatwpmi.gov · birchlaketownshipmn.gov · ausabletwp-mi.gov · bishophillil.gov · bluefieldwv.gov · bethelparkpa.gov · arcadami.gov · bloomingdaletownshipmi.gov · bloomingdalevillagemi.gov · bogotanj.gov · bogotapolicenj.gov · blanding-ut.gov · bloomingdale-ga.gov · brackettvilletx.gov · bradfordme.gov · bridgeportne.gov · brinkhaven.gov · bristoltownshipohio.gov · brooklyntownwashburnwi.gov · bucklinmo.gov · burnham-il.gov
Look up any .gov domain
Method and limits
Population: CISA dotgov-data current-full.csv (all registered .gov domains with type, organization, city and state). Fetch: one HTTPS request to the apex (then www.), following redirects, 8-second timeout, user agent agentexchange-census/1.0; robots.txt, /.well-known/security.txt, /security.txt, /llms.txt and /.well-known/agent-card.json on the final host. "Reachable" means any HTTP response over HTTPS. Homepage links are detected from anchor targets and visible text, so a policy that exists but is not linked from the homepage is not counted. A robots.txt "blocked" verdict means the agent's group (or *) disallows the whole site. One snapshot; municipal sites that were slow or that block automated requests are counted as unreachable. Data: JSON (CC BY 4.0).
Run a .gov site? Generate a security.txt robots.txt rules for AI crawlers · llms.txt generator · accessibility feedback desk (free)