{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreibsgtf5bkxp5vlhhciaskd6ezw5ozoqkg7crhj6trpzd3qjxxf7ae",
    "uri": "at://did:plc:25rdn5elo5izoxrmtis34zuk/app.bsky.feed.post/3mpk7h6d77fe2"
  },
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreigyffutndnn33nhh6rzrerr2lfr6y5r5zeodwngsa6ylfwzfghhnq"
    },
    "mimeType": "image/webp",
    "size": 66482
  },
  "path": "/scrapemint/how-to-screen-thousands-of-stocks-without-a-data-provider-125h",
  "publishedAt": "2026-06-30T23:11:24.000Z",
  "site": "https://dev.to",
  "tags": [
    "webdev",
    "api",
    "javascript",
    "showdev"
  ],
  "textContent": "Stock screeners feel like something you should have to pay a data vendor for. Market cap, sector, P/E, dividend yield, RSI, daily change, all of it across thousands of tickers, updated through the day. In practice a lot of that is one keyless JSON call away, and you can run it yourself.\n\n##  The scanner most people scroll past\n\nTradingView's screener page is backed by a public endpoint that takes a filter and returns matching symbols:\n\n\n\n    POST https://scanner.tradingview.com/america/scan\n\n\nThe body is just a filter, the columns you want, a sort, and a range:\n\n\n\n    {\n      \"filter\": [\n        { \"left\": \"market_cap_basic\", \"operation\": \"egreater\", \"right\": 10000000000 },\n        { \"left\": \"dividend_yield_recent\", \"operation\": \"egreater\", \"right\": 3 }\n      ],\n      \"columns\": [\"name\", \"close\", \"dividend_yield_recent\", \"market_cap_basic\", \"RSI\", \"sector\"],\n      \"sort\": { \"sortBy\": \"market_cap_basic\", \"sortOrder\": \"desc\" },\n      \"range\": [0, 100]\n    }\n\n\nYou get back a `totalCount` and a `data` array, one entry per symbol, with a `d` array of values lined up to the columns you asked for. No login, no API key, no headless browser. The same endpoint serves other markets too: swap `america` for `crypto`, `forex`, `india`, `uk`, and more.\n\n##  Push the filter down, do not pull the table\n\nThe mistake I see most often is pulling a giant table and filtering it in your own code. That is slow, and it ships data you never wanted. The scanner filters server side, so you only receive the rows that already match. Want large cap names that are oversold? Send `market_cap_basic` greater than ten billion and `RSI` less than thirty, and that is all that comes back.\n\nA few useful column ids, since they are not obvious:\n\n  * `close` is the last price\n  * `change` is the percent change on the day\n  * `market_cap_basic` is market cap\n  * `price_earnings_ttm` is trailing P/E\n  * `dividend_yield_recent` is the dividend yield\n  * `Perf.YTD` is year to date performance\n\n\n\nOperations you will reach for: `egreater` and `eless` for greater or equal and less or equal, `in_range` for a band or a set of values, and `greater` and `less` for strict bounds. `in_range` doubles as a set membership test, so a list of sectors works the same way as a numeric band.\n\n##  Paging and cost\n\nEach request takes a `range`, so paging is just moving the window: `[0,100]`, then `[100,200]`, until you hit `totalCount` or the cap you set. Because there is no browser and no proxy, a full screen of a few hundred symbols costs almost nothing to run. Normalize the numbers once when they come in and every downstream join behaves.\n\n##  I packaged it\n\nI turned this into a small Actor on Apify so it is callable from code without writing the request by hand. You set friendly filters like market cap, sector, P/E, dividend yield, RSI or daily change, and it returns one tidy row per symbol with price, change, volume, market cap, P/E, dividend yield, RSI, sector, industry, exchange, country and YTD. It is the newest of a growing set of keyless finance scrapers I have been shipping. The first rows of every run are free so you can check the output before you scale up.\n\nThe wider point stands on its own though: before you reach for a paid data plan, check whether the site you already trust is serving the same numbers over a public endpoint. Often it is.",
  "title": "How to Screen Thousands of Stocks Without a Data Provider"
}