--- title: "Scraping SEC EDGAR vs a SEC Filings API: What AI Agents Actually Pay" slug: scraping-sec-edgar-vs-sec-filings-api date: 2026-07-13 excerpt: "EDGAR is free and its JSON APIs are real — so when does a SEC filings API earn its keep? An honest cost breakdown for teams wiring AI agents to SEC filings." category: AI & Agents tags: ["AI & Agents", "Data Infrastructure"] cta: filings author: Jennifer Ma authorRole: "Co-founder & CEO" status: published --- Every build-vs-buy debate about a SEC filings API starts from a true premise: SEC EDGAR is free, authoritative, and — fewer people know this — ships real JSON APIs of its own. So before we sell you anything, let's give the free route its full due, then put numbers on where it stops. The short version: scraping SEC EDGAR is the right call more often than vendors admit, and a SEC filings API earns its keep at a specific, predictable point — the moment agents need *narrative* content, at fleet scale, with citations that survive an audit. **TLDR:** - EDGAR's free APIs are genuinely good for two things: the per-company filing index ([updated with sub-second delay](https://www.sec.gov/search-filings/edgar-application-programming-interfaces)) and standardized XBRL numbers. - They stop exactly where agent workloads begin: narrative sections exist only as raw HTML, transcripts aren't on EDGAR at all, and everything shares a [10 requests/second cap](https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data). - The cost of bridging that gap yourself is measured, not hypothetical: [81% failure rates from naive retrieval](https://arxiv.org/abs/2311.11944) and [decision flips in 21–53% of free-text summaries](https://arxiv.org/abs/2606.29251). ## What the free route actually gives you Credit where due — [data.sec.gov](https://www.sec.gov/search-filings/edgar-application-programming-interfaces) hosts RESTful JSON APIs covering the submissions history of every filer and the XBRL data from financial statements (10-K, 10-Q, 8-K, 20-F, 40-F, 6-K and variants), and they're fast: | Free EDGAR API | What it returns | Freshness | | --- | --- | --- | | `submissions/` | Per-company filing history + entity metadata (names, tickers, exchanges) | < 1 second typical delay | | `xbrl/companyfacts` | All XBRL concepts for one company in one call | < 1 minute | | `xbrl/companyconcept` | One concept across a company's filings | < 1 minute | | `xbrl/frames` | One concept across all filers, per period | < 1 minute | | Bulk ZIP downloads | Every JSON structure above, for offline datasets | Refreshed daily | *Source: [SEC.gov — EDGAR Application Programming Interfaces](https://www.sec.gov/search-filings/edgar-application-programming-interfaces). As of July 2026.* If your agent needs to know *when* a company filed and what its standardized financial line items were, the free tier is legitimately excellent — [about 4,700 filings flow through EDGAR every day](https://www.sec.gov/submit-filings/about-edgar) and the index reflects them in real time. ## Where the free route stops Now the other half of the map. The SEC's own scope statement is precise: the APIs contain the submissions index and XBRL financial data. Everything an analyst — or an analyst agent — reads for *judgment* lives outside them: ![Two-panel diagram: EDGAR's free APIs cover the filing index, XBRL numbers, bulk downloads, and search; they don't cover narrative sections as data, transcripts, entity normalization, or a citation layer](/images/blog/scraping-sec-edgar-vs-sec-filings-api/fig-1.svg) *Figure 1: The free API surface vs what agent workloads need. Source: [SEC.gov](https://www.sec.gov/search-filings/edgar-application-programming-interfaces).* | What agents need | Status in EDGAR's free APIs | | --- | --- | | Narrative sections as data (MD&A, Risk Factors, footnotes) | Not included — raw HTML in the document archives, parsed by you | | Earnings-call transcripts | Not SEC filings; not on EDGAR at all | | Horizontal queries across filers (e.g. all funds holding one ticker) | Not included — 13F reports are per fund; [33,739 13F-HRs filed in 2025](https://www.sec.gov/data-research/sec-markets-data/number-edgar-filings-form-type) to invert yourself | | KPIs, segment detail, custom-tagged items | Standard-taxonomy facts only — [frames aggregates only non-custom taxonomy facts](https://www.sec.gov/search-filings/edgar-application-programming-interfaces); KPIs live in releases and calls | | Entity normalization (names, segments, executives across filers) | Ticker/exchange metadata only | | A citation layer for extracted values | Not included — provenance is yours to rebuild | | Point-in-time / as-amended data | Not resolved — [1,239 amended 13F-HRs and 708 amended 10-Ks in 2025](https://www.sec.gov/data-research/sec-markets-data/number-edgar-filings-form-type) sit as separate filings to dedupe | | Throughput for an agent fleet | Shared [10 req/s cap per user, all machines combined](https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data) | | Browser-side access | [No CORS support](https://www.sec.gov/search-filings/edgar-application-programming-interfaces) | *Sources: [SEC.gov, EDGAR APIs](https://www.sec.gov/search-filings/edgar-application-programming-interfaces); [SEC.gov, Accessing EDGAR Data](https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data); [SEC DERA, filings by form type](https://www.sec.gov/data-research/sec-markets-data/number-edgar-filings-form-type).* Three of those rows deserve a beat. **Horizontal queries**: EDGAR answers "what did this fund file?" in one document, but not *"which funds hold this ticker, and what changed?"* — that means parsing all ~8,400 information tables each quarter into your own inverted index. A data layer offers both directions as single calls ([holdings by investor](/docs/api/institutional-holdings/holdings-by-investor), [owners by ticker](/docs/api/institutional-holdings/owners-by-ticker)), and the same normalization lets an agent pull one company across quarters or compare companies side by side without hand-stitching filer-chosen tags. **KPIs**: the metrics that move stocks — subscribers, deliveries, segment splits — are either custom-tagged (which the SEC says [reduces comparability](https://www.sec.gov/data-research/gaap-xbrl-custom-tags) and the cross-company [frames API excludes](https://www.sec.gov/search-filings/edgar-application-programming-interfaces)) or never tagged, living in releases and calls; a data layer extracts them into cited fields ([segment financials](/docs/api/financials/segments), [documents](/docs/api/documents)). **Point-in-time**: filers restate, so a naive scrape can serve a number the filer later corrected — resolving as-amended and preserving point-in-time snapshots is the data layer's job, not the agent's. Closing the remaining gaps yourself is a parsing-normalization-citation pipeline, and its failure modes have been measured: | DIY shortcut over raw filings | Measured cost | | --- | --- | | Casual retrieval pipeline (vector store over filings) | GPT-4-Turbo fails 81% of questions — vs 21% given the right documents | | Free-text summarization of 10-Q MD&A | Flips the investment decision in 21.3–53.0% of cases (11.0% baseline) | | Whole documents dumped into long context | All 18 frontier models tested degrade as input grows | *Sources: [FinanceBench, arXiv:2311.11944](https://arxiv.org/abs/2311.11944) with [Patronus AI](https://www.patronus.ai/announcements/patronus-ai-launches-financebench-the-industrys-first-benchmark-for-llm-performance-on-financial-questions); [When Summaries Distort Decisions, arXiv:2606.29251](https://arxiv.org/html/2606.29251); [Context Rot, Chroma Research](https://www.trychroma.com/research/context-rot).* We've unpacked each of these in the [evaluation-checklist pillar](/blog/sec-filings-api-for-ai-agents) and the [compression-fidelity post](/blog/when-ai-compresses-your-filings). ## When scraping is the right call Honesty clause. You do *not* need a SEC filings API when: - The work is occasional, single-company research — a human in the loop, a handful of documents. - You only need standardized XBRL numerics — the free APIs plus [bulk ZIPs](https://www.sec.gov/search-filings/edgar-application-programming-interfaces) are built for exactly this. - You're prototyping and learning the domain — reading raw filings teaches you what normalization has to solve. Stay inside the [fair-access rules](https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data) (≤10 req/s, declared User-Agent — sec.gov 403s anonymous fetchers, as we rediscovered researching this post) and the free route is stable and dependable. ## When the API pays for itself The switch flips when three things are simultaneously true: it's *agents* doing the reading (no human catching parsing garbage), they need *narrative* content (where the [decontextualization risk](/blog/when-ai-compresses-your-filings) lives), and their answers must be *auditable* (every value traceable to its source document). That's the configuration where DIY pipelines quietly convert infrastructure debt into wrong answers — and where a data layer's normalization, section-level retrieval, and built-in citations stop being conveniences and become the product. ![Decision flow: occasional lookups — scrape or use free APIs; XBRL numbers only — free APIs plus bulk files; agent fleets needing narrative sections, citations, and audit trails — a SEC filings API](/images/blog/scraping-sec-edgar-vs-sec-filings-api/fig-2.svg) *Figure 2: The decision, compressed. All direct sec.gov access stays subject to the fair-access policy.* FocusAlpha is built for that third branch: a [SEC filings API](/docs/api/filings) where filings arrive normalized, sections are individually retrievable, every value keeps its citation, transcripts and other company communications ship through the same interface, and agents connect over [API or MCP](/docs). For how to evaluate us — or anyone — use [the seven-point checklist](/blog/sec-filings-api-for-ai-agents). ## FAQ ### Does SEC EDGAR have a free API? Yes. [data.sec.gov](https://www.sec.gov/search-filings/edgar-application-programming-interfaces) offers free JSON APIs: per-company submissions history (typical delay under a second) and XBRL financial-statement data (companyfacts, companyconcept, frames; under a minute), plus daily bulk ZIP downloads. ### What's the difference between EDGAR's free APIs and a commercial SEC filings API? EDGAR's APIs return the filing index and standardized XBRL numbers, organized per filer. A commercial SEC filings API adds what agent workloads need on top: narrative sections as structured data, entity normalization, source citations on every value, transcripts, throughput beyond the shared [10 req/s cap](https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data) — and horizontal queries EDGAR can't express, like [all funds holding a ticker](/docs/api/institutional-holdings/owners-by-ticker) or one company's [normalized statements across quarters](/docs/api/financials). ### How fresh is EDGAR data? Near real-time: the SEC states the submissions API updates with [a typical processing delay of under a second, and the XBRL APIs under a minute](https://www.sec.gov/search-filings/edgar-application-programming-interfaces), as filings are disseminated throughout the day. ### Is scraping SEC EDGAR allowed? Yes, within the [fair-access policy](https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data): at most 10 requests per second per user across all machines, with a declared User-Agent identifying you. Exceed it and your IP is temporarily blocked; anonymous fetchers get HTTP 403. ### When should an AI agent scrape EDGAR directly instead of using a SEC filings API? When the workload is occasional single-company research, when only standardized XBRL numerics are needed, or when you're prototyping. The free APIs and bulk files handle those well — the calculus changes at fleet scale on narrative content with audit requirements. ### When does a SEC filings API pay off? When agents read narrative sections (MD&A, risk factors, transcripts) at scale and their answers must be auditable. The measured alternative is expensive: [81% failure from naive retrieval](https://arxiv.org/abs/2311.11944) and [21–53% decision flips from free-text summarization](https://arxiv.org/abs/2606.29251). ### Does EDGAR's XBRL data include company KPIs like subscribers or deliveries? Generally no. Operating KPIs and non-GAAP metrics live in earnings releases, presentations, and calls rather than the standardized taxonomy — and EDGAR's cross-company frames API [aggregates only non-custom taxonomy facts](https://www.sec.gov/search-filings/edgar-application-programming-interfaces), excluding the custom tags the SEC says [reduce inter-company comparability](https://www.sec.gov/data-research/gaap-xbrl-custom-tags). Getting KPIs as structured data means extracting them from the documents themselves, the way a data layer's [segmented financials](/docs/api/financials/segments) and [documents](/docs/api/documents) endpoints do. ### What is FocusAlpha? FocusAlpha is a [SEC filings API](/docs/api/filings) and agent-ready financial data layer: it turns SEC filings (10-K, 10-Q, 8-K, 13F), earnings-call transcripts, and other trusted company communications into structured, normalized data where every value keeps its citation back to the source document. AI agents connect via [API or MCP](/docs) to research public companies from complete, trusted information. ## Sources - [SEC.gov — EDGAR Application Programming Interfaces (data.sec.gov scope, endpoints, freshness, CORS)](https://www.sec.gov/search-filings/edgar-application-programming-interfaces) - [SEC.gov — Accessing EDGAR Data (fair-access policy: 10 req/s, User-Agent)](https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data) - [SEC.gov — About EDGAR (≈4,700 filings/day)](https://www.sec.gov/submit-filings/about-edgar) - [SEC DERA — Number of EDGAR Filings by Form Type (13F-HR counts)](https://www.sec.gov/data-research/sec-markets-data/number-edgar-filings-form-type) - [SEC DERA — U.S. GAAP XBRL Custom Tags Trend (comparability acknowledgment)](https://www.sec.gov/data-research/gaap-xbrl-custom-tags) - [FinanceBench — arXiv:2311.11944](https://arxiv.org/abs/2311.11944) - [Patronus AI — FinanceBench announcement](https://www.patronus.ai/announcements/patronus-ai-launches-financebench-the-industrys-first-benchmark-for-llm-performance-on-financial-questions) - [When Summaries Distort Decisions — arXiv:2606.29251](https://arxiv.org/abs/2606.29251) - [Context Rot — Chroma Research](https://www.trychroma.com/research/context-rot)