How to Scrape Reddit Subreddit Posts in 2026

Reddit is the largest archive of unfiltered human opinion on the open web. For social listening, brand monitoring, and market research, there’s no substitute for the candor of a subreddit thread. The catch: Reddit’s official API has become expensive and rate-limited since the 2023 pricing changes, and most teams don’t want to register an OAuth app just to read public posts. This guide covers what works in 2026 — the public JSON endpoints Reddit still exposes, what data you can pull, and how to extract it cleanly across many subreddits at once.

The public JSON endpoint nobody talks about

Reddit’s website is a React app, but under the hood almost every listing page has a JSON twin. Append .json to any subreddit listing URL and you get the structured payload the front-end uses:

https://www.reddit.com/r/programming/new.json?limit=100
https://www.reddit.com/r/programming/top.json?t=week&limit=100
https://www.reddit.com/r/programming/hot.json
https://www.reddit.com/r/programming/rising.json

This is the key insight that makes account-free scraping viable. You don’t need the official oauth.reddit.com API, you don’t need a client ID, and you don’t need a developer app. The four sort orders — new, hot, top, rising — each return a Listing object with a children array of posts and an after cursor for pagination.

The t parameter (hour, day, week, month, year, all) only applies to the top listing and controls the time window. This matters a lot: top.json?t=all gives you the all-time greatest hits of a community, while new.json gives you the live firehose.

What the payload actually contains

Each post in the children array is a deeply nested object. The fields worth flattening into a per-post record:

• Identity — post ID (name, the t3_ fullname), permalink, subreddit.
• Content — title, selftext (the body for text posts), url (the link for link posts).
• Engagement — score, upvote_ratio, num_comments.
• Author — author display name.
• Flags — over_18 (NSFW), is_video, stickied, link_flair_text.
• Timing — created_utc (epoch seconds, convert to ISO).

The nesting is the annoying part. Reddit wraps every post in { "kind": "t3", "data": { ... } }, and the listing wraps everything in { "data": { "children": [...], "after": "..." } }. A good scraper flattens all of this into clean, one-row-per-post records so you don’t write the same unwrapping code for the hundredth time.

▶ Run the Reddit Subreddit Scraper — pulls posts from any subreddit (multiple per run) with title, author, score, comments, flair, body text and timestamps. No Reddit account or API key needed.

The two things that break naive scrapers

Hitting reddit.com/r/X.json from a script looks easy in a browser, but two things bite at scale.

1. Datacenter IP blocks. Reddit aggressively rate-limits and blocks requests from cloud ASNs (AWS, GCP, Hetzner). A handful of requests from a fresh datacenter IP and you’ll start getting 429 Too Many Requests or an empty body. The fix is residential proxy routing — requests that look like they come from real home connections. A managed scraper rotates these automatically so you never see the block.

2. Pagination cursors, not page numbers. Reddit doesn’t paginate with ?page=2. It uses an opaque after token returned in each response, which you feed into the next request as ?after=t3_abc123. Miss the cursor handling and you’ll re-scrape page one forever, or stop after 100 posts. Full pagination traversal means following the cursor until after comes back null.

There’s also a soft cap: Reddit’s listings only go about 1,000 posts deep per sort order regardless of cursors. For high-volume communities, the trick is combining sort orders and time windows — top?t=year, top?t=month, new — to maximize unique coverage.

Schema design for downstream use

Whatever you do with the data, normalize it on the way in. A clean per-post schema:

{
  "id": "t3_1abc234",
  "subreddit": "programming",
  "title": "Why we migrated off Kubernetes",
  "selftext": "After three years running our stack on...",
  "url": "https://www.reddit.com/r/programming/comments/1abc234/...",
  "author": "throwaway_devops",
  "score": 1842,
  "upvote_ratio": 0.94,
  "num_comments": 376,
  "link_flair_text": "Discussion",
  "is_video": false,
  "over_18": false,
  "stickied": false,
  "created_utc": "2026-05-18T09:14:00Z",
  "scraped_at": "2026-05-20T12:00:00Z"
}

A few choices worth making early:

• Keep both score and upvote_ratio. A post at 500 score with a 0.55 ratio is controversial; one at 500 with 0.97 is loved. The ratio is half the signal.
• Store selftext and url separately. Text posts and link posts are different beasts; collapsing them loses information.
• Always stamp scraped_at. Scores climb for hours after posting. You need to know when a snapshot was valid to track velocity.
• Keep the t3_ fullname as the join key. Titles get edited; the fullname never changes.

Typical use cases

What people actually do with subreddit data:

• Brand and competitor monitoring — scrape industry and category subreddits on a schedule, surface mentions, sentiment, and complaint volume before they hit Twitter.
• Community and audience research — discover the actual language a niche uses, the products they recommend, the pain points they repeat.
• Lead generation — watch for-hire and “looking for a tool that does X” subreddits for fresh demand posts.
• Financial and crypto sentiment — track ticker mentions and trend momentum across r/wallstreetbets-style communities for alpha signals.
• Content and trend discovery — pull top-performing posts to seed newsletters, articles, and creator content.
• NLP / LLM corpora — build domain-targeted text datasets from niche subreddits for fine-tuning.

The common thread is recency. A snapshot of r/SaaS from last month is mostly noise; a daily-refreshed feed across 30 relevant subreddits is a monitoring system.

Cost math

Running this as a managed actor on Apify is pay-per-result, and for Reddit JSON listings the per-post cost is effectively negligible — the expensive parts (residential proxy bandwidth and pagination logic) are bundled in. A realistic monitoring setup — 30 subreddits, new plus top?t=day, refreshed daily — lands around 3,000–6,000 posts per run. Over a month that’s well within Apify’s free monthly credit for most users, and into the low single digits of dollars for heavier pulls.

Compare that to building it yourself:

• A residential proxy pool you can actually scale on runs $200–400/month.
• The cursor-pagination and JSON-unwrapping code is a day to write and a recurring chore to keep correct when Reddit tweaks its payload shape.
• IP-block whack-a-mole eats an afternoon every few weeks.

For a monitoring workload, the proxy bill alone dwarfs the managed cost.

Common pitfalls

• Don’t confuse posts with comments. This pulls posts (the t3 objects). Comment trees live at a different endpoint (/comments/<id>.json) and are a separate scraping problem.
• Respect the 1,000-post depth cap. If you need deeper history, combine sort orders and time windows rather than fighting the cursor.
• Watch for [deleted] and [removed]. Removed posts still appear in listings with stub content. Filter them if you’re building a clean corpus.
• NSFW gating. Some subreddits require a logged-in over-18 confirmation; the public JSON may return thinner data for those. Flag over_18 and handle accordingly.
• Timezone math. created_utc is epoch seconds in UTC. Convert once, store ISO, and never debug a “posts are 3 hours off” ticket again.

Wrapping up

If you need a one-time pull from a single subreddit, the .json endpoint plus a residential proxy is a fun afternoon project. If you need a refreshed feed across dozens of communities for monitoring or research, the maintenance — proxy pool, cursor pagination, payload-shape drift — adds up fast. A managed actor solves all three once and keeps them solved.

▶ Open the Reddit Subreddit Scraper on Apify — multi-subreddit, all four sort orders, time-window control, flat export-ready schema. No API key. Start with Apify’s free monthly credit.