Scraping Social Media Data with Python: X (Twitter), Reddit & Instagram (2026)

Over 500 million posts are published on X (formerly Twitter) every single day. Reddit hosts more than 100,000 active communities discussing everything from machine learning to local weather. Instagram's 2 billion users generate a continuous stream of product opinions, lifestyle signals, and trend indicators.

Social media data is the closest thing the internet has to real-time public opinion. It powers:

• Sentiment analysis — What do people think about your product, brand, or competitor right now?

• Trend detection — What topics are accelerating before they go mainstream?

• Academic research — Studying misinformation, political discourse, crisis communication

• Market intelligence — Tracking competitor mentions, industry conversations

• AI training datasets — Social text is gold for fine-tuning conversational models

The challenge in 2026: platforms have made scraping harder than ever. The official X API now starts at $100/month and the free tier is almost useless for research. Instagram aggressively blocks headless browsers. Reddit has rate-limited its API severely after the 2023 controversy.

This guide gives you working approaches for all three platforms — from DIY Python scrapers to managed tools — with honest assessments of what each method can and cannot do.

The 2026 Reality Check: What's Still Scrapable

Before writing a single line of code, here's the honest state of social media scraping in 2026:

Key principles:

• Never scrape private data, DMs, or anything behind a login you don't own

• Always respect robots.txt and Terms of Service for your use case

• For academic or commercial research, official APIs or licensed providers are the safer path

Part 1: Scraping X (Twitter) in 2026

The Landscape

X's public guest API was effectively removed in 2023. Almost every endpoint now requires a logged-in session and a CSRF token. The internal GraphQL API that underpins X's own web interface is the main DIY scraping target — but its endpoint identifiers change regularly.

Three working approaches exist in 2026:

Method 1: X API v2 (Official)

For authorised projects, the official API is always the right choice. The free tier gives 500k tweets/month read access:

pip install tweepy

import tweepy
import pandas as pd
from datetime import datetime, timezone, timedelta

# Get credentials at developer.twitter.com
client = tweepy.Client(bearer_token="YOUR_BEARER_TOKEN")

def search_recent_tweets(
    query: str,
    max_results: int = 100,
    days_back: int = 7
) -> pd.DataFrame:
    """
    Search tweets from the last 7 days using X API v2.

    Args:
        query: Search query. Supports operators like:
               - AND/OR: "python scraping OR web crawling"
               - Exact: '"machine learning" tutorial'
               - Exclude: "python -is:retweet"
               - Language: "python lang:en"
               - Has media: "python has:images"
    """
    start_time = datetime.now(timezone.utc) - timedelta(days=days_back)

    # Append filters to reduce noise
    full_query = f"{query} -is:retweet lang:en"

    tweets_data = []
    paginator = tweepy.Paginator(
        client.search_recent_tweets,
        query=full_query,
        tweet_fields=[
            "created_at", "public_metrics", "author_id",
            "lang", "context_annotations", "entities"
        ],
        user_fields=["name", "username", "public_metrics", "verified"],
        expansions=["author_id"],
        start_time=start_time,
        max_results=min(100, max_results),
    )

    users_map = {}

    for response in paginator:
        if not response.data:
            break

        # Build user lookup from includes
        if response.includes and "users" in response.includes:
            for user in response.includes["users"]:
                users_map[user.id] = user

        for tweet in response.data:
            user = users_map.get(tweet.author_id)
            metrics = tweet.public_metrics or {}

            tweets_data.append({
                "tweet_id":       str(tweet.id),
                "text":           tweet.text,
                "created_at":     tweet.created_at,
                "author_id":      str(tweet.author_id),
                "username":       user.username if user else None,
                "name":           user.name if user else None,
                "followers":      user.public_metrics.get("followers_count") if user else None,
                "retweets":       metrics.get("retweet_count", 0),
                "likes":          metrics.get("like_count", 0),
                "replies":        metrics.get("reply_count", 0),
                "quotes":         metrics.get("quote_count", 0),
                "impressions":    metrics.get("impression_count", 0),
                "url":            f"https://x.com/{user.username if user else 'i'}/status/{tweet.id}",
            })

        if len(tweets_data) >= max_results:
            break

    df = pd.DataFrame(tweets_data)
    print(f"Collected {len(df)} tweets for query: '{query}'")
    return df


# Example: track brand mentions
df = search_recent_tweets(
    query="python web scraping 2026",
    max_results=200,
    days_back=7
)
df.to_csv("tweets.csv", index=False)
print(df[["username", "text", "likes", "retweets"]].head(10))

Method 2: Playwright-Based X Scraper (No API Key)

For personal research without API access, Playwright can scrape public search results and profiles using a saved session — exactly as described for LinkedIn in Blog 02:

import asyncio
import random
import json
from playwright.async_api import async_playwright
from playwright_stealth import stealth_async

async def save_x_session():
    """Log in to X manually once and save cookies. Run this first."""
    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=False)
        context = await browser.new_context(
            user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120"
        )
        page = await context.new_page()
        await page.goto("https://x.com/login")

        print("Log in manually, then press Enter...")
        input()

        cookies = await context.cookies()
        with open("x_cookies.json", "w") as f:
            json.dump(cookies, f)
        print(f"Saved {len(cookies)} cookies.")
        await browser.close()


async def scrape_x_search(query: str, scroll_times: int = 10) -> list[dict]:
    """
    Scrape X search results using a saved session.
    Returns a list of tweet dicts.
    """
    async with async_playwright() as p:
        browser = await p.chromium.launch(
            headless=True,
            args=["--disable-blink-features=AutomationControlled"]
        )
        context = await browser.new_context(
            user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120",
            viewport={"width": 1280, "height": 900}
        )

        # Load saved session
        with open("x_cookies.json") as f:
            await context.add_cookies(json.load(f))

        page = await context.new_page()
        await stealth_async(page)

        # Navigate to search
        encoded = query.replace(" ", "%20")
        await page.goto(
            f"https://x.com/search?q={encoded}&src=typed_query&f=top",
            wait_until="domcontentloaded"
        )
        await asyncio.sleep(random.uniform(2, 4))

        tweets = []
        seen_ids = set()

        for scroll in range(scroll_times):
            # Extract tweet data from the current viewport
            tweet_articles = await page.query_selector_all("article[data-testid='tweet']")

            for article in tweet_articles:
                try:
                    # Get tweet text
                    text_el = await article.query_selector("[data-testid='tweetText']")
                    text = await text_el.inner_text() if text_el else None

                    # Username
                    user_el = await article.query_selector("[data-testid='User-Name'] a")
                    href = await user_el.get_attribute("href") if user_el else ""
                    username = href.strip("/").split("/")[-1] if href else None

                    # Display name
                    name_els = await article.query_selector_all(
                        "[data-testid='User-Name'] span"
                    )
                    display_name = None
                    for el in name_els:
                        txt = await el.inner_text()
                        if txt and not txt.startswith("@"):
                            display_name = txt
                            break

                    # Engagement stats
                    async def get_stat(testid):
                        el = await article.query_selector(f"[data-testid='{testid}']")
                        if el:
                            txt = await el.inner_text()
                            return txt.strip() or "0"
                        return "0"

                    likes    = await get_stat("like")
                    replies  = await get_stat("reply")
                    retweets = await get_stat("retweet")

                    # Time
                    time_el = await article.query_selector("time")
                    posted_at = await time_el.get_attribute("datetime") if time_el else None

                    # Unique ID to deduplicate
                    tweet_id = f"{username}_{posted_at}"
                    if tweet_id in seen_ids or not text:
                        continue
                    seen_ids.add(tweet_id)

                    tweets.append({
                        "username":     username,
                        "display_name": display_name,
                        "text":         text,
                        "likes":        likes,
                        "replies":      replies,
                        "retweets":     retweets,
                        "posted_at":    posted_at,
                    })
                except Exception:
                    continue

            # Scroll down for more tweets
            await page.evaluate("window.scrollBy(0, window.innerHeight * 1.5)")
            await asyncio.sleep(random.uniform(1.5, 3.0))
            print(f"  Scroll {scroll+1}/{scroll_times} — {len(tweets)} tweets collected")

        await browser.close()

    print(f"\nTotal unique tweets: {len(tweets)}")
    return tweets


# Run
tweets = asyncio.run(scrape_x_search("python scraping 2026", scroll_times=8))
df = pd.DataFrame(tweets)
df.to_csv("x_search_results.csv", index=False)

Part 2: Reddit Scraping with PRAW

Reddit is the most developer-friendly major social platform for data collection. Its official Python wrapper PRAW (Python Reddit API Wrapper) is free, well-documented, and gives access to posts, comments, user profiles, and subreddit data at no cost.

pip install praw pandas

Register a Reddit app at reddit.com/prefs/apps (takes 2 minutes, free) to get your client ID and secret.

Scraping subreddit posts

import praw
import pandas as pd
from datetime import datetime, timezone

reddit = praw.Reddit(
    client_id="YOUR_CLIENT_ID",
    client_secret="YOUR_CLIENT_SECRET",
    user_agent="PythonResearchBot/1.0 by u/your_username",
)

def scrape_subreddit(
    subreddit_name: str,
    sort: str = "hot",         # "hot", "new", "top", "rising"
    time_filter: str = "week", # "hour", "day", "week", "month", "year", "all"
    limit: int = 500
) -> pd.DataFrame:
    """
    Scrape posts from a subreddit.

    Best for: topic research, trend monitoring, content analysis.
    """
    sub = reddit.subreddit(subreddit_name)

    # Choose sort method
    if sort == "hot":
        posts_gen = sub.hot(limit=limit)
    elif sort == "new":
        posts_gen = sub.new(limit=limit)
    elif sort == "top":
        posts_gen = sub.top(time_filter=time_filter, limit=limit)
    elif sort == "rising":
        posts_gen = sub.rising(limit=limit)
    else:
        posts_gen = sub.hot(limit=limit)

    records = []
    for post in posts_gen:
        records.append({
            "post_id":       post.id,
            "title":         post.title,
            "text":          post.selftext[:500] if post.selftext else None,
            "author":        str(post.author) if post.author else "[deleted]",
            "score":         post.score,
            "upvote_ratio":  post.upvote_ratio,
            "num_comments":  post.num_comments,
            "url":           post.url,
            "permalink":     f"https://reddit.com{post.permalink}",
            "flair":         post.link_flair_text,
            "is_self":       post.is_self,
            "created_utc":   datetime.fromtimestamp(post.created_utc, tz=timezone.utc).isoformat(),
            "subreddit":     subreddit_name,
        })

    df = pd.DataFrame(records)
    print(f"Scraped {len(df)} posts from r/{subreddit_name}")
    return df


# Example: research Python discussions
df = scrape_subreddit("learnpython", sort="top", time_filter="month", limit=500)
df.to_csv("reddit_learnpython.csv", index=False)
print(df[["title", "score", "num_comments"]].head(10))

Scraping post comments (deep dive)

def scrape_post_comments(
    post_url: str,
    max_comments: int = 200,
    include_replies: bool = False
) -> pd.DataFrame:
    """
    Scrape all comments from a Reddit post.
    Useful for sentiment analysis, topic deep-dives, building datasets.
    """
    submission = reddit.submission(url=post_url)

    # Replace "MoreComments" objects to get all comments
    submission.comments.replace_more(limit=0)

    all_comments = submission.comments.list() if include_replies else submission.comments

    records = []
    for comment in list(all_comments)[:max_comments]:
        if not hasattr(comment, "body"):
            continue
        records.append({
            "comment_id":    comment.id,
            "author":        str(comment.author) if comment.author else "[deleted]",
            "body":          comment.body,
            "score":         comment.score,
            "depth":         comment.depth,
            "created_utc":   datetime.fromtimestamp(
                                 comment.created_utc, tz=timezone.utc
                             ).isoformat(),
            "is_op":         comment.is_submitter,
            "awards":        comment.total_awards_received,
        })

    return pd.DataFrame(records)


# Example: deep dive on a specific post
comments_df = scrape_post_comments(
    "https://www.reddit.com/r/MachineLearning/comments/example/",
    max_comments=300
)
print(f"Top comment: {comments_df.sort_values('score', ascending=False).iloc[0]['body'][:200]}")

Multi-subreddit keyword monitoring

def monitor_keyword_across_subreddits(
    keyword: str,
    subreddits: list[str],
    limit_per_sub: int = 100
) -> pd.DataFrame:
    """
    Search for a keyword across multiple subreddits.
    Great for competitive intelligence and brand monitoring.
    """
    all_posts = []

    for sub_name in subreddits:
        print(f"Searching r/{sub_name} for '{keyword}'...")
        try:
            sub = reddit.subreddit(sub_name)
            for post in sub.search(keyword, sort="new", limit=limit_per_sub):
                all_posts.append({
                    "subreddit":    sub_name,
                    "title":        post.title,
                    "text":         post.selftext[:300] if post.selftext else None,
                    "author":       str(post.author) if post.author else "[deleted]",
                    "score":        post.score,
                    "comments":     post.num_comments,
                    "permalink":    f"https://reddit.com{post.permalink}",
                    "created_utc":  datetime.fromtimestamp(
                                        post.created_utc, tz=timezone.utc
                                    ).isoformat(),
                })
        except Exception as e:
            print(f"  Error on r/{sub_name}: {e}")

    df = pd.DataFrame(all_posts)

    # Summary by subreddit
    summary = df.groupby("subreddit").agg(
        posts=("title", "count"),
        avg_score=("score", "mean"),
        total_comments=("comments", "sum")
    ).sort_values("posts", ascending=False)

    print(f"\n── Results for '{keyword}' ──")
    print(summary.to_string())

    return df


# Track "Python scraping" across tech subreddits
df = monitor_keyword_across_subreddits(
    keyword="python scraping",
    subreddits=["Python", "learnpython", "webdev", "datascience", "MachineLearning"],
    limit_per_sub=50
)
df.to_csv("reddit_keyword_monitor.csv", index=False)

Part 3: Instagram Scraping (Public Profiles Only)

Instagram is the most aggressively protected major platform in 2026. The public guest API was removed years ago. Almost all data requires a logged-in session. The GraphQL API underlying Instagram's web interface changes identifiers constantly.

This section covers only public profile and hashtag data — the bare minimum available without login — and the Playwright approach for logged-in access to your own account's data.

Scraping public profile metadata

import httpx
import json
import asyncio

async def get_instagram_profile(username: str) -> dict | None:
    """
    Fetch public profile data for an Instagram username.
    Uses the semi-public shared data endpoint.
    Only works for public accounts.
    """
    url = f"https://www.instagram.com/{username}/?__a=1&__d=dis"

    headers = {
        "User-Agent": (
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
            "AppleWebKit/537.36 Chrome/120 Safari/537.36"
        ),
        "Accept": "*/*",
        "Referer": "https://www.instagram.com/",
        "X-IG-App-ID": "936619743392459",  # Instagram Web app ID
    }

    async with httpx.AsyncClient(headers=headers, follow_redirects=True) as client:
        try:
            r = await client.get(url, timeout=15)
            if r.status_code == 200:
                data = r.json()
                user = data.get("graphql", {}).get("user", {})
                return {
                    "username":       user.get("username"),
                    "full_name":      user.get("full_name"),
                    "biography":      user.get("biography"),
                    "followers":      user.get("edge_followed_by", {}).get("count"),
                    "following":      user.get("edge_follow", {}).get("count"),
                    "posts":          user.get("edge_owner_to_timeline_media", {}).get("count"),
                    "is_verified":    user.get("is_verified"),
                    "is_business":    user.get("is_business_account"),
                    "profile_url":    f"https://www.instagram.com/{username}/",
                }
        except Exception as e:
            print(f"Error fetching @{username}: {e}")
    return None


# Batch scrape public profiles
async def batch_profile_scrape(usernames: list[str]) -> pd.DataFrame:
    results = []
    for username in usernames:
        print(f"Fetching @{username}...")
        profile = await get_instagram_profile(username)
        if profile:
            results.append(profile)
        await asyncio.sleep(random.uniform(2, 5))
    return pd.DataFrame(results)


profiles_df = asyncio.run(batch_profile_scrape([
    "natgeo", "nasa", "python.learning"
]))
print(profiles_df[["username", "followers", "posts", "is_verified"]])

Playwright-based Instagram scraper (logged-in session)

For your own account's data or influencer research on public accounts:

import asyncio, json, random
from playwright.async_api import async_playwright
from playwright_stealth import stealth_async

async def save_instagram_session():
    """Log in manually and save session cookies. Run once."""
    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=False)
        context = await browser.new_context(
            user_agent="Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) "
                       "AppleWebKit/605.1.15 Mobile/15E148 Safari/604.1",
            viewport={"width": 390, "height": 844},
            is_mobile=True,
        )
        page = await context.new_page()
        await page.goto("https://www.instagram.com/accounts/login/")
        print("Log in manually, then press Enter...")
        input()
        cookies = await context.cookies()
        with open("ig_cookies.json", "w") as f:
            json.dump(cookies, f)
        print("Session saved.")
        await browser.close()


async def scrape_instagram_posts(username: str, max_posts: int = 30) -> list[dict]:
    """
    Scrape recent posts from a public Instagram profile.
    Requires a saved session from save_instagram_session().
    """
    async with async_playwright() as p:
        browser = await p.chromium.launch(
            headless=True,
            args=["--no-sandbox", "--disable-blink-features=AutomationControlled"]
        )
        context = await browser.new_context(
            user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120",
            viewport={"width": 1280, "height": 900}
        )
        with open("ig_cookies.json") as f:
            await context.add_cookies(json.load(f))

        page = await context.new_page()
        await stealth_async(page)

        await page.goto(
            f"https://www.instagram.com/{username}/",
            wait_until="domcontentloaded"
        )
        await asyncio.sleep(random.uniform(2, 4))

        posts = []
        # Scroll to load posts
        for _ in range(max_posts // 12 + 1):
            post_links = await page.query_selector_all("article a[href*='/p/']")
            for link in post_links:
                href = await link.get_attribute("href")
                if href and href not in [p.get("href") for p in posts]:
                    posts.append({"href": href, "username": username})
            await page.evaluate("window.scrollBy(0, window.innerHeight)")
            await asyncio.sleep(random.uniform(1, 2.5))

        # Deduplicate and limit
        seen = set()
        unique_posts = []
        for p in posts:
            if p["href"] not in seen:
                seen.add(p["href"])
                unique_posts.append(p)
        posts = unique_posts[:max_posts]

        await browser.close()

    print(f"Found {len(posts)} posts for @{username}")
    return posts

Part 4: Sentiment Analysis on Collected Data

Once you've collected social media data, analysing sentiment turns raw text into actionable signals. The transformers library from HuggingFace provides excellent pre-trained sentiment models:

pip install transformers torch

from transformers import pipeline
import pandas as pd

# Load a pre-trained sentiment model (downloads ~67MB on first run)
# "cardiffnlp/twitter-roberta-base-sentiment-latest" is trained on tweets
sentiment_pipeline = pipeline(
    "sentiment-analysis",
    model="cardiffnlp/twitter-roberta-base-sentiment-latest",
    tokenizer="cardiffnlp/twitter-roberta-base-sentiment-latest",
    max_length=512,
    truncation=True,
)

LABEL_MAP = {
    "LABEL_0": "negative",
    "LABEL_1": "neutral",
    "LABEL_2": "positive",
}

def analyse_sentiment_batch(texts: list[str], batch_size: int = 32) -> list[dict]:
    """
    Run sentiment analysis on a list of social media texts.
    Returns list of {label, score} dicts.
    """
    results = []
    for i in range(0, len(texts), batch_size):
        batch = texts[i:i + batch_size]
        # Clean texts — remove URLs and excessive whitespace
        cleaned = [
            " ".join(word for word in t.split() if not word.startswith("http"))[:512]
            for t in batch
        ]
        preds = sentiment_pipeline(cleaned)
        for pred in preds:
            results.append({
                "sentiment": LABEL_MAP.get(pred["label"], pred["label"]),
                "confidence": round(pred["score"], 3),
            })
        print(f"  Processed {min(i + batch_size, len(texts))}/{len(texts)}")

    return results


def sentiment_report(df: pd.DataFrame, text_col: str = "text") -> pd.DataFrame:
    """
    Enrich a DataFrame with sentiment scores and produce a summary report.
    """
    texts = df[text_col].fillna("").tolist()
    print(f"Analysing sentiment for {len(texts)} posts...")
    sentiments = analyse_sentiment_batch(texts)

    df["sentiment"]   = [s["sentiment"]  for s in sentiments]
    df["confidence"]  = [s["confidence"] for s in sentiments]

    # Summary
    summary = df["sentiment"].value_counts(normalize=True).mul(100).round(1)
    print("\n── Sentiment Distribution ──")
    for label, pct in summary.items():
        bar = "█" * int(pct / 2)
        print(f"  {label:10s}: {bar} {pct}%")

    return df


# Apply to Reddit data
reddit_df = pd.read_csv("reddit_learnpython.csv")
reddit_df = sentiment_report(reddit_df, text_col="title")
reddit_df.to_csv("reddit_with_sentiment.csv", index=False)

# Apply to X data
tweets_df = pd.read_csv("tweets.csv")
tweets_df = sentiment_report(tweets_df, text_col="text")
tweets_df.to_csv("tweets_with_sentiment.csv", index=False)

Part 5: Building a Brand Monitoring Dashboard

Combine all sources into a unified brand monitoring pipeline:

import asyncio
import pandas as pd
from datetime import datetime, timezone

async def run_brand_monitor(brand_name: str, competitor: str = None) -> dict:
    """
    Collect brand mentions from Reddit and X, analyse sentiment,
    and produce a unified brand health report.
    """
    report = {
        "brand":       brand_name,
        "run_at":      datetime.now(timezone.utc).isoformat(),
        "reddit":      {},
        "twitter":     {},
        "sentiment":   {},
    }

    # ── Reddit ────────────────────────────────────────────────
    print(f"\n[1/3] Scraping Reddit for '{brand_name}'...")
    reddit_df = monitor_keyword_across_subreddits(
        keyword=brand_name,
        subreddits=["technology", "Python", "webdev", "datascience", "programming"],
        limit_per_sub=50
    )
    reddit_df = sentiment_report(reddit_df, text_col="title")

    report["reddit"] = {
        "total_posts":     len(reddit_df),
        "avg_score":       round(reddit_df["score"].mean(), 1),
        "total_comments":  int(reddit_df["comments"].sum()),
        "top_post":        reddit_df.sort_values("score", ascending=False).iloc[0]["title"],
        "top_subreddit":   reddit_df["subreddit"].value_counts().index[0],
    }

    # ── Sentiment ─────────────────────────────────────────────
    all_sentiments = pd.concat([
        reddit_df[["sentiment", "confidence"]],
    ])
    sentiment_counts = all_sentiments["sentiment"].value_counts(normalize=True).mul(100)
    report["sentiment"] = {
        "positive_pct": round(sentiment_counts.get("positive", 0), 1),
        "neutral_pct":  round(sentiment_counts.get("neutral", 0), 1),
        "negative_pct": round(sentiment_counts.get("negative", 0), 1),
        "overall":      sentiment_counts.idxmax(),
    }

    # ── Print Report ──────────────────────────────────────────
    print(f"\n{'═'*50}")
    print(f"  BRAND MONITOR: {brand_name.upper()}")
    print(f"  Run at: {report['run_at']}")
    print(f"{'═'*50}")
    print(f"\n  Reddit mentions:   {report['reddit']['total_posts']}")
    print(f"  Avg post score:    {report['reddit']['avg_score']}")
    print(f"  Most active sub:   r/{report['reddit']['top_subreddit']}")
    print(f"\n  Sentiment:")
    print(f"    ✅ Positive:  {report['sentiment']['positive_pct']}%")
    print(f"    😐 Neutral:   {report['sentiment']['neutral_pct']}%")
    print(f"    ❌ Negative:  {report['sentiment']['negative_pct']}%")
    print(f"    Overall:      {report['sentiment']['overall'].upper()}")
    print(f"\n  Top post: \"{report['reddit']['top_post'][:80]}...\"")

    return report


# Run
report = asyncio.run(run_brand_monitor("python scraping"))

Rate Limits and Platform Rules Reference

FAQ

Q: snscrape is broken in 2026 — what should I use instead? snscrape's X scraper broke in 2023 when X removed the guest API and has not been reliably fixed. Use Tweepy with X API v2 for authorised projects. For DIY scraping, use the Playwright approach above.

Q: Can I scrape private Instagram posts? No — and you shouldn't. Scraping private account data without consent violates Instagram's Terms of Service, GDPR, and potentially criminal hacking laws in many jurisdictions. Only collect publicly available data.

Q: What's the best model for social media sentiment analysis in 2026?cardiffnlp/twitter-roberta-base-sentiment-latest is trained specifically on tweets and outperforms generic models. For Reddit, which is longer-form, distilbert-base-uncased-finetuned-sst-2-english works well. For multilingual content, use nlptown/bert-base-multilingual-uncased-sentiment.

Q: Reddit changed its API — is PRAW still free? Yes. PRAW's free tier (500 requests per 10 minutes) was not affected by the 2023 API changes. Only third-party apps accessing certain endpoints at very high volume were impacted. Standard research use via PRAW remains free.

Q: How do I handle deleted posts in my Reddit dataset? Posts with [deleted] author or [removed] body are common. Filter them before analysis: df = df[df['author'] != '[deleted]'] and df = df[df['text'] != '[removed]'].

Summary