{"schemaVersion":"1.0","type":"Article","types":["Article"],"slug":"scraping-social-media-data-with-python-x-twitter-reddit-instagram-2026-k7og3","url":"https://api.zyvop.com/scraping-social-media-data-with-python-x-twitter-reddit-instagram-2026-k7og3","title":"Scraping Social Media Data with Python: X (Twitter), Reddit & Instagram (2026)","subtitle":"Learn how to scrape X (Twitter), Reddit, and Instagram with Python in 2026. Working code for tweets, subreddit posts, profiles, and sentiment analysis — with anti-bot handling.","tldr":"Discover practical techniques for collecting public social media data, handling anti-bot protections, analyzing sentiment, and building a unified monitoring pipeline for research, analytics, and brand intelligence.","keywords":["python social media scraping tutorial 2026","scrape twitter x python 2026","scrape reddit python tutorial","instagram data scraping python","social media sentiment analysis python scraping","Scraping"],"entities":["ZyVOP","python social media scraping tutorial 2026","scrape twitter x python 2026","scrape reddit python tutorial","instagram data scraping python","social media sentiment analysis python scraping","Scraping"],"keyTakeaways":["Platform Best Method Key Library What You Can Get X / Twitter Tweepy (API v2) tweepy Tweets, metrics, author data X / Twitter Playwright + session playwright Search results, profiles Reddit PRAW praw Posts, comments, subreddits Instagram httpx + cookies httpx Public profiles, post counts All platforms HuggingFace transformers Sentiment, emotion, topics."],"headings":["The 2026 Reality Check: What's Still Scrapable","Part 1: Scraping X (Twitter) in 2026","The Landscape","Method 1: X API v2 (Official)","Method 2: Playwright-Based X Scraper (No API Key)","Part 2: Reddit Scraping with PRAW","Scraping subreddit posts","Scraping post comments (deep dive)","Multi-subreddit keyword monitoring","Part 3: Instagram Scraping (Public Profiles Only)","Scraping public profile metadata","Playwright-based Instagram scraper (logged-in session)","Part 4: Sentiment Analysis on Collected Data","Part 5: Building a Brand Monitoring Dashboard","Rate Limits and Platform Rules Reference","FAQ","Summary"],"outboundLinks":["https://reddit.com/prefs/apps"],"contentText":"Over 500 million posts are published on X (formerly Twitter) every single day. Reddit hosts more than 100,000 active communities discussing everything from machine learning to local weather. Instagram's 2 billion users generate a continuous stream of product opinions, lifestyle signals, and trend indicators. Social media data is the closest thing the internet has to real-time public opinion. It powers: Sentiment analysis — What do people think about your product, brand, or competitor right now? Trend detection — What topics are accelerating before they go mainstream? Academic research — Studying misinformation, political discourse, crisis communication Market intelligence — Tracking competitor mentions, industry conversations AI training datasets — Social text is gold for fine-tuning conversational models The challenge in 2026: platforms have made scraping harder than ever. The official X API now starts at $100/month and the free tier is almost useless for research. Instagram aggressively blocks headless browsers. Reddit has rate-limited its API severely after the 2023 controversy. This guide gives you working approaches for all three platforms — from DIY Python scrapers to managed tools — with honest assessments of what each method can and cannot do. The 2026 Reality Check: What's Still Scrapable Before writing a single line of code, here's the honest state of social media scraping in 2026: Platform Public Posts Profile Data Followers DMs Difficulty X / Twitter ✅ (with effort) ✅ ✅ ❌ Hard Reddit ✅ (via API/PRAW) ✅ ❌ ❌ Easy–Medium Instagram ⚠️ Public only ⚠️ Public only ❌ ❌ Very Hard TikTok ✅ (via mobile API) ✅ ❌ ❌ Hard LinkedIn ⚠️ (see Blog 02) ⚠️ (see Blog 02) ❌ ❌ Hard Key principles: Never scrape private data, DMs, or anything behind a login you don't own Always respect robots.txt and Terms of Service for your use case For academic or commercial research, official APIs or licensed providers are the safer path Part 1: Scraping X (Twitter) in 2026 The Landscape X's public guest API was effectively removed in 2023. Almost every endpoint now requires a logged-in session and a CSRF token. The internal GraphQL API that underpins X's own web interface is the main DIY scraping target — but its endpoint identifiers change regularly. Three working approaches exist in 2026: Method Cost Volume Reliability Best for X API v2 (official) $100+/month Limited High Authorised developers Playwright + session Free Low-medium Medium Research, personal use ScrapeGraphAI Paid per req High High Production Method 1: X API v2 (Official) For authorised projects, the official API is always the right choice. The free tier gives 500k tweets/month read access: pip install tweepyimport tweepy import pandas as pd from datetime import datetime, timezone, timedelta # Get credentials at developer.twitter.com client = tweepy.Client(bearer_token=\"YOUR_BEARER_TOKEN\") def search_recent_tweets( query: str, max_results: int = 100, days_back: int = 7 ) -> pd.DataFrame: \"\"\" Search tweets from the last 7 days using X API v2. Args: query: Search query. Supports operators like: - AND/OR: \"python scraping OR web crawling\" - Exact: '\"machine learning\" tutorial' - Exclude: \"python -is:retweet\" - Language: \"python lang:en\" - Has media: \"python has:images\" \"\"\" start_time = datetime.now(timezone.utc) - timedelta(days=days_back) # Append filters to reduce noise full_query = f\"{query} -is:retweet lang:en\" tweets_data = [] paginator = tweepy.Paginator( client.search_recent_tweets, query=full_query, tweet_fields=[ \"created_at\", \"public_metrics\", \"author_id\", \"lang\", \"context_annotations\", \"entities\" ], user_fields=[\"name\", \"username\", \"public_metrics\", \"verified\"], expansions=[\"author_id\"], start_time=start_time, max_results=min(100, max_results), ) users_map = {} for response in paginator: if not response.data: break # Build user lookup from includes if response.includes and \"users\" in response.includes: for user in response.includes[\"users\"]: users_map[user.id] = user for tweet in response.data: user = users_map.get(tweet.author_id) metrics = tweet.public_metrics or {} tweets_data.append({ \"tweet_id\": str(tweet.id), \"text\": tweet.text, \"created_at\": tweet.created_at, \"author_id\": str(tweet.author_id), \"username\": user.username if user else None, \"name\": user.name if user else None, \"followers\": user.public_metrics.get(\"followers_count\") if user else None, \"retweets\": metrics.get(\"retweet_count\", 0), \"likes\": metrics.get(\"like_count\", 0), \"replies\": metrics.get(\"reply_count\", 0), \"quotes\": metrics.get(\"quote_count\", 0), \"impressions\": metrics.get(\"impression_count\", 0), \"url\": f\"https://x.com/{user.username if user else 'i'}/status/{tweet.id}\", }) if len(tweets_data) >= max_results: break df = pd.DataFrame(tweets_data) print(f\"Collected {len(df)} tweets for query: '{query}'\") return df # Example: track brand mentions df = search_recent_tweets( query=\"python web scraping 2026\", max_results=200, days_back=7 ) df.to_csv(\"tweets.csv\", index=False) print(df[[\"username\", \"text\", \"likes\", \"retweets\"]].head(10)) Method 2: Playwright-Based X Scraper (No API Key) For personal research without API access, Playwright can scrape public search results and profiles using a saved session — exactly as described for LinkedIn in Blog 02: import asyncio import random import json from playwright.async_api import async_playwright from playwright_stealth import stealth_async async def save_x_session(): \"\"\"Log in to X manually once and save cookies. Run this first.\"\"\" async with async_playwright() as p: browser = await p.chromium.launch(headless=False) context = await browser.new_context( user_agent=\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120\" ) page = await context.new_page() await page.goto(\"https://x.com/login\") print(\"Log in manually, then press Enter...\") input() cookies = await context.cookies() with open(\"x_cookies.json\", \"w\") as f: json.dump(cookies, f) print(f\"Saved {len(cookies)} cookies.\") await browser.close() async def scrape_x_search(query: str, scroll_times: int = 10) -> list[dict]: \"\"\" Scrape X search results using a saved session. Returns a list of tweet dicts. \"\"\" async with async_playwright() as p: browser = await p.chromium.launch( headless=True, args=[\"--disable-blink-features=AutomationControlled\"] ) context = await browser.new_context( user_agent=\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120\", viewport={\"width\": 1280, \"height\": 900} ) # Load saved session with open(\"x_cookies.json\") as f: await context.add_cookies(json.load(f)) page = await context.new_page() await stealth_async(page) # Navigate to search encoded = query.replace(\" \", \"%20\") await page.goto( f\"https://x.com/search?q={encoded}&src=typed_query&f=top\", wait_until=\"domcontentloaded\" ) await asyncio.sleep(random.uniform(2, 4)) tweets = [] seen_ids = set() for scroll in range(scroll_times): # Extract tweet data from the current viewport tweet_articles = await page.query_selector_all(\"article[data-testid='tweet']\") for article in tweet_articles: try: # Get tweet text text_el = await article.query_selector(\"[data-testid='tweetText']\") text = await text_el.inner_text() if text_el else None # Username user_el = await article.query_selector(\"[data-testid='User-Name'] a\") href = await user_el.get_attribute(\"href\") if user_el else \"\" username = href.strip(\"/\").split(\"/\")[-1] if href else None # Display name name_els = await article.query_selector_all( \"[data-testid='User-Name'] span\" ) display_name = None for el in name_els: txt = await el.inner_text() if txt and not txt.startswith(\"@\"): display_name = txt break # Engagement stats async def get_stat(testid): el = await article.query_selector(f\"[data-testid='{testid}']\") if el: txt = await el.inner_text() return txt.strip() or \"0\" return \"0\" likes = await get_stat(\"like\") replies = await get_stat(\"reply\") retweets = await get_stat(\"retweet\") # Time time_el = await article.query_selector(\"time\") posted_at = await time_el.get_attribute(\"datetime\") if time_el else None # Unique ID to deduplicate tweet_id = f\"{username}_{posted_at}\" if tweet_id in seen_ids or not text: continue seen_ids.add(tweet_id) tweets.append({ \"username\": username, \"display_name\": display_name, \"text\": text, \"likes\": likes, \"replies\": replies, \"retweets\": retweets, \"posted_at\": posted_at, }) except Exception: continue # Scroll down for more tweets await page.evaluate(\"window.scrollBy(0, window.innerHeight * 1.5)\") await asyncio.sleep(random.uniform(1.5, 3.0)) print(f\" Scroll {scroll+1}/{scroll_times} — {len(tweets)} tweets collected\") await browser.close() print(f\"\\nTotal unique tweets: {len(tweets)}\") return tweets # Run tweets = asyncio.run(scrape_x_search(\"python scraping 2026\", scroll_times=8)) df = pd.DataFrame(tweets) df.to_csv(\"x_search_results.csv\", index=False) Part 2: Reddit Scraping with PRAW Reddit is the most developer-friendly major social platform for data collection. Its official Python wrapper PRAW (Python Reddit API Wrapper) is free, well-documented, and gives access to posts, comments, user profiles, and subreddit data at no cost. pip install praw pandas Register a Reddit app at reddit.com/prefs/apps (takes 2 minutes, free) to get your client ID and secret. Scraping subreddit posts import praw import pandas as pd from datetime import datetime, timezone reddit = praw.Reddit( client_id=\"YOUR_CLIENT_ID\", client_secret=\"YOUR_CLIENT_SECRET\", user_agent=\"PythonResearchBot/1.0 by u/your_username\", ) def scrape_subreddit( subreddit_name: str, sort: str = \"hot\", # \"hot\", \"new\", \"top\", \"rising\" time_filter: str = \"week\", # \"hour\", \"day\", \"week\", \"month\", \"year\", \"all\" limit: int = 500 ) -> pd.DataFrame: \"\"\" Scrape posts from a subreddit. Best for: topic research, trend monitoring, content analysis. \"\"\" sub = reddit.subreddit(subreddit_name) # Choose sort method if sort == \"hot\": posts_gen = sub.hot(limit=limit) elif sort == \"new\": posts_gen = sub.new(limit=limit) elif sort == \"top\": posts_gen = sub.top(time_filter=time_filter, limit=limit) elif sort == \"rising\": posts_gen = sub.rising(limit=limit) else: posts_gen = sub.hot(limit=limit) records = [] for post in posts_gen: records.append({ \"post_id\": post.id, \"title\": post.title, \"text\": post.selftext[:500] if post.selftext else None, \"author\": str(post.author) if post.author else \"[deleted]\", \"score\": post.score, \"upvote_ratio\": post.upvote_ratio, \"num_comments\": post.num_comments, \"url\": post.url, \"permalink\": f\"https://reddit.com{post.permalink}\", \"flair\": post.link_flair_text, \"is_self\": post.is_self, \"created_utc\": datetime.fromtimestamp(post.created_utc, tz=timezone.utc).isoformat(), \"subreddit\": subreddit_name, }) df = pd.DataFrame(records) print(f\"Scraped {len(df)} posts from r/{subreddit_name}\") return df # Example: research Python discussions df = scrape_subreddit(\"learnpython\", sort=\"top\", time_filter=\"month\", limit=500) df.to_csv(\"reddit_learnpython.csv\", index=False) print(df[[\"title\", \"score\", \"num_comments\"]].head(10)) Scraping post comments (deep dive) def scrape_post_comments( post_url: str, max_comments: int = 200, include_replies: bool = False ) -> pd.DataFrame: \"\"\" Scrape all comments from a Reddit post. Useful for sentiment analysis, topic deep-dives, building datasets. \"\"\" submission = reddit.submission(url=post_url) # Replace \"MoreComments\" objects to get all comments submission.comments.replace_more(limit=0) all_comments = submission.comments.list() if include_replies else submission.comments records = [] for comment in list(all_comments)[:max_comments]: if not hasattr(comment, \"body\"): continue records.append({ \"comment_id\": comment.id, \"author\": str(comment.author) if comment.author else \"[deleted]\", \"body\": comment.body, \"score\": comment.score, \"depth\": comment.depth, \"created_utc\": datetime.fromtimestamp( comment.created_utc, tz=timezone.utc ).isoformat(), \"is_op\": comment.is_submitter, \"awards\": comment.total_awards_received, }) return pd.DataFrame(records) # Example: deep dive on a specific post comments_df = scrape_post_comments( \"https://www.reddit.com/r/MachineLearning/comments/example/\", max_comments=300 ) print(f\"Top comment: {comments_df.sort_values('score', ascending=False).iloc[0]['body'][:200]}\") Multi-subreddit keyword monitoring def monitor_keyword_across_subreddits( keyword: str, subreddits: list[str], limit_per_sub: int = 100 ) -> pd.DataFrame: \"\"\" Search for a keyword across multiple subreddits. Great for competitive intelligence and brand monitoring. \"\"\" all_posts = [] for sub_name in subreddits: print(f\"Searching r/{sub_name} for '{keyword}'...\") try: sub = reddit.subreddit(sub_name) for post in sub.search(keyword, sort=\"new\", limit=limit_per_sub): all_posts.append({ \"subreddit\": sub_name, \"title\": post.title, \"text\": post.selftext[:300] if post.selftext else None, \"author\": str(post.author) if post.author else \"[deleted]\", \"score\": post.score, \"comments\": post.num_comments, \"permalink\": f\"https://reddit.com{post.permalink}\", \"created_utc\": datetime.fromtimestamp( post.created_utc, tz=timezone.utc ).isoformat(), }) except Exception as e: print(f\" Error on r/{sub_name}: {e}\") df = pd.DataFrame(all_posts) # Summary by subreddit summary = df.groupby(\"subreddit\").agg( posts=(\"title\", \"count\"), avg_score=(\"score\", \"mean\"), total_comments=(\"comments\", \"sum\") ).sort_values(\"posts\", ascending=False) print(f\"\\n── Results for '{keyword}' ──\") print(summary.to_string()) return df # Track \"Python scraping\" across tech subreddits df = monitor_keyword_across_subreddits( keyword=\"python scraping\", subreddits=[\"Python\", \"learnpython\", \"webdev\", \"datascience\", \"MachineLearning\"], limit_per_sub=50 ) df.to_csv(\"reddit_keyword_monitor.csv\", index=False) Part 3: Instagram Scraping (Public Profiles Only) Instagram is the most aggressively protected major platform in 2026. The public guest API was removed years ago. Almost all data requires a logged-in session. The GraphQL API underlying Instagram's web interface changes identifiers constantly. This section covers only public profile and hashtag data — the bare minimum available without login — and the Playwright approach for logged-in access to your own account's data. Scraping public profile metadata import httpx import json import asyncio async def get_instagram_profile(username: str) -> dict | None: \"\"\" Fetch public profile data for an Instagram username. Uses the semi-public shared data endpoint. Only works for public accounts. \"\"\" url = f\"https://www.instagram.com/{username}/?__a=1&__d=dis\" headers = { \"User-Agent\": ( \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) \" \"AppleWebKit/537.36 Chrome/120 Safari/537.36\" ), \"Accept\": \"*/*\", \"Referer\": \"https://www.instagram.com/\", \"X-IG-App-ID\": \"936619743392459\", # Instagram Web app ID } async with httpx.AsyncClient(headers=headers, follow_redirects=True) as client: try: r = await client.get(url, timeout=15) if r.status_code == 200: data = r.json() user = data.get(\"graphql\", {}).get(\"user\", {}) return { \"username\": user.get(\"username\"), \"full_name\": user.get(\"full_name\"), \"biography\": user.get(\"biography\"), \"followers\": user.get(\"edge_followed_by\", {}).get(\"count\"), \"following\": user.get(\"edge_follow\", {}).get(\"count\"), \"posts\": user.get(\"edge_owner_to_timeline_media\", {}).get(\"count\"), \"is_verified\": user.get(\"is_verified\"), \"is_business\": user.get(\"is_business_account\"), \"profile_url\": f\"https://www.instagram.com/{username}/\", } except Exception as e: print(f\"Error fetching @{username}: {e}\") return None # Batch scrape public profiles async def batch_profile_scrape(usernames: list[str]) -> pd.DataFrame: results = [] for username in usernames: print(f\"Fetching @{username}...\") profile = await get_instagram_profile(username) if profile: results.append(profile) await asyncio.sleep(random.uniform(2, 5)) return pd.DataFrame(results) profiles_df = asyncio.run(batch_profile_scrape([ \"natgeo\", \"nasa\", \"python.learning\" ])) print(profiles_df[[\"username\", \"followers\", \"posts\", \"is_verified\"]]) Playwright-based Instagram scraper (logged-in session) For your own account's data or influencer research on public accounts: import asyncio, json, random from playwright.async_api import async_playwright from playwright_stealth import stealth_async async def save_instagram_session(): \"\"\"Log in manually and save session cookies. Run once.\"\"\" async with async_playwright() as p: browser = await p.chromium.launch(headless=False) context = await browser.new_context( user_agent=\"Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) \" \"AppleWebKit/605.1.15 Mobile/15E148 Safari/604.1\", viewport={\"width\": 390, \"height\": 844}, is_mobile=True, ) page = await context.new_page() await page.goto(\"https://www.instagram.com/accounts/login/\") print(\"Log in manually, then press Enter...\") input() cookies = await context.cookies() with open(\"ig_cookies.json\", \"w\") as f: json.dump(cookies, f) print(\"Session saved.\") await browser.close() async def scrape_instagram_posts(username: str, max_posts: int = 30) -> list[dict]: \"\"\" Scrape recent posts from a public Instagram profile. Requires a saved session from save_instagram_session(). \"\"\" async with async_playwright() as p: browser = await p.chromium.launch( headless=True, args=[\"--no-sandbox\", \"--disable-blink-features=AutomationControlled\"] ) context = await browser.new_context( user_agent=\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120\", viewport={\"width\": 1280, \"height\": 900} ) with open(\"ig_cookies.json\") as f: await context.add_cookies(json.load(f)) page = await context.new_page() await stealth_async(page) await page.goto( f\"https://www.instagram.com/{username}/\", wait_until=\"domcontentloaded\" ) await asyncio.sleep(random.uniform(2, 4)) posts = [] # Scroll to load posts for _ in range(max_posts // 12 + 1): post_links = await page.query_selector_all(\"article a[href*='/p/']\") for link in post_links: href = await link.get_attribute(\"href\") if href and href not in [p.get(\"href\") for p in posts]: posts.append({\"href\": href, \"username\": username}) await page.evaluate(\"window.scrollBy(0, window.innerHeight)\") await asyncio.sleep(random.uniform(1, 2.5)) # Deduplicate and limit seen = set() unique_posts = [] for p in posts: if p[\"href\"] not in seen: seen.add(p[\"href\"]) unique_posts.append(p) posts = unique_posts[:max_posts] await browser.close() print(f\"Found {len(posts)} posts for @{username}\") return posts Part 4: Sentiment Analysis on Collected Data Once you've collected social media data, analysing sentiment turns raw text into actionable signals. The transformers library from HuggingFace provides excellent pre-trained sentiment models: pip install transformers torch from transformers import pipeline import pandas as pd # Load a pre-trained sentiment model (downloads ~67MB on first run) # \"cardiffnlp/twitter-roberta-base-sentiment-latest\" is trained on tweets sentiment_pipeline = pipeline( \"sentiment-analysis\", model=\"cardiffnlp/twitter-roberta-base-sentiment-latest\", tokenizer=\"cardiffnlp/twitter-roberta-base-sentiment-latest\", max_length=512, truncation=True, ) LABEL_MAP = { \"LABEL_0\": \"negative\", \"LABEL_1\": \"neutral\", \"LABEL_2\": \"positive\", } def analyse_sentiment_batch(texts: list[str], batch_size: int = 32) -> list[dict]: \"\"\" Run sentiment analysis on a list of social media texts. Returns list of {label, score} dicts. \"\"\" results = [] for i in range(0, len(texts), batch_size): batch = texts[i:i + batch_size] # Clean texts — remove URLs and excessive whitespace cleaned = [ \" \".join(word for word in t.split() if not word.startswith(\"http\"))[:512] for t in batch ] preds = sentiment_pipeline(cleaned) for pred in preds: results.append({ \"sentiment\": LABEL_MAP.get(pred[\"label\"], pred[\"label\"]), \"confidence\": round(pred[\"score\"], 3), }) print(f\" Processed {min(i + batch_size, len(texts))}/{len(texts)}\") return results def sentiment_report(df: pd.DataFrame, text_col: str = \"text\") -> pd.DataFrame: \"\"\" Enrich a DataFrame with sentiment scores and produce a summary report. \"\"\" texts = df[text_col].fillna(\"\").tolist() print(f\"Analysing sentiment for {len(texts)} posts...\") sentiments = analyse_sentiment_batch(texts) df[\"sentiment\"] = [s[\"sentiment\"] for s in sentiments] df[\"confidence\"] = [s[\"confidence\"] for s in sentiments] # Summary summary = df[\"sentiment\"].value_counts(normalize=True).mul(100).round(1) print(\"\\n── Sentiment Distribution ──\") for label, pct in summary.items(): bar = \"█\" * int(pct / 2) print(f\" {label:10s}: {bar} {pct}%\") return df # Apply to Reddit data reddit_df = pd.read_csv(\"reddit_learnpython.csv\") reddit_df = sentiment_report(reddit_df, text_col=\"title\") reddit_df.to_csv(\"reddit_with_sentiment.csv\", index=False) # Apply to X data tweets_df = pd.read_csv(\"tweets.csv\") tweets_df = sentiment_report(tweets_df, text_col=\"text\") tweets_df.to_csv(\"tweets_with_sentiment.csv\", index=False) Part 5: Building a Brand Monitoring Dashboard Combine all sources into a unified brand monitoring pipeline: import asyncio import pandas as pd from datetime import datetime, timezone async def run_brand_monitor(brand_name: str, competitor: str = None) -> dict: \"\"\" Collect brand mentions from Reddit and X, analyse sentiment, and produce a unified brand health report. \"\"\" report = { \"brand\": brand_name, \"run_at\": datetime.now(timezone.utc).isoformat(), \"reddit\": {}, \"twitter\": {}, \"sentiment\": {}, } # ── Reddit ──────────────────────────────────────────────── print(f\"\\n[1/3] Scraping Reddit for '{brand_name}'...\") reddit_df = monitor_keyword_across_subreddits( keyword=brand_name, subreddits=[\"technology\", \"Python\", \"webdev\", \"datascience\", \"programming\"], limit_per_sub=50 ) reddit_df = sentiment_report(reddit_df, text_col=\"title\") report[\"reddit\"] = { \"total_posts\": len(reddit_df), \"avg_score\": round(reddit_df[\"score\"].mean(), 1), \"total_comments\": int(reddit_df[\"comments\"].sum()), \"top_post\": reddit_df.sort_values(\"score\", ascending=False).iloc[0][\"title\"], \"top_subreddit\": reddit_df[\"subreddit\"].value_counts().index[0], } # ── Sentiment ───────────────────────────────────────────── all_sentiments = pd.concat([ reddit_df[[\"sentiment\", \"confidence\"]], ]) sentiment_counts = all_sentiments[\"sentiment\"].value_counts(normalize=True).mul(100) report[\"sentiment\"] = { \"positive_pct\": round(sentiment_counts.get(\"positive\", 0), 1), \"neutral_pct\": round(sentiment_counts.get(\"neutral\", 0), 1), \"negative_pct\": round(sentiment_counts.get(\"negative\", 0), 1), \"overall\": sentiment_counts.idxmax(), } # ── Print Report ────────────────────────────────────────── print(f\"\\n{'═'*50}\") print(f\" BRAND MONITOR: {brand_name.upper()}\") print(f\" Run at: {report['run_at']}\") print(f\"{'═'*50}\") print(f\"\\n Reddit mentions: {report['reddit']['total_posts']}\") print(f\" Avg post score: {report['reddit']['avg_score']}\") print(f\" Most active sub: r/{report['reddit']['top_subreddit']}\") print(f\"\\n Sentiment:\") print(f\" ✅ Positive: {report['sentiment']['positive_pct']}%\") print(f\" 😐 Neutral: {report['sentiment']['neutral_pct']}%\") print(f\" ❌ Negative: {report['sentiment']['negative_pct']}%\") print(f\" Overall: {report['sentiment']['overall'].upper()}\") print(f\"\\n Top post: \\\"{report['reddit']['top_post'][:80]}...\\\"\") return report # Run report = asyncio.run(run_brand_monitor(\"python scraping\")) Rate Limits and Platform Rules Reference Platform Daily free limit Rate limit reset Key restriction X API v2 (Free) 500k tweets/month read 15 min windows No historical data X API v2 (Basic, $100/mo) 10M tweets/month 15 min windows 30-day history Reddit PRAW 1,000 req/10 min Rolling Public posts only Instagram (no auth) ~200 req/hour Hourly Public profiles only Instagram (with session) 200–500 actions/day Daily Avoid aggressive scraping FAQ Q: snscrape is broken in 2026 — what should I use instead? snscrape's X scraper broke in 2023 when X removed the guest API and has not been reliably fixed. Use Tweepy with X API v2 for authorised projects. For DIY scraping, use the Playwright approach above. Q: Can I scrape private Instagram posts? No — and you shouldn't. Scraping private account data without consent violates Instagram's Terms of Service, GDPR, and potentially criminal hacking laws in many jurisdictions. Only collect publicly available data. Q: What's the best model for social media sentiment analysis in 2026?cardiffnlp/twitter-roberta-base-sentiment-latest is trained specifically on tweets and outperforms generic models. For Reddit, which is longer-form, distilbert-base-uncased-finetuned-sst-2-english works well. For multilingual content, use nlptown/bert-base-multilingual-uncased-sentiment. Q: Reddit changed its API — is PRAW still free? Yes. PRAW's free tier (500 requests per 10 minutes) was not affected by the 2023 API changes. Only third-party apps accessing certain endpoints at very high volume were impacted. Standard research use via PRAW remains free. Q: How do I handle deleted posts in my Reddit dataset? Posts with [deleted] author or [removed] body are common. Filter them before analysis: df = df[df['author'] != '[deleted]'] and df = df[df['text'] != '[removed]']. Summary Platform Best Method Key Library What You Can Get X / Twitter Tweepy (API v2) tweepy Tweets, metrics, author data X / Twitter Playwright + session playwright Search results, profiles Reddit PRAW praw Posts, comments, subreddits Instagram httpx + cookies httpx Public profiles, post counts All platforms HuggingFace transformers Sentiment, emotion, topics","contentHash":"sha256:e11d6dac64e4d30ed4e6128125ed116893af649d05bcbd76f3a9de35938d228b","authorName":"ZyVOP","authorUrl":"https://api.zyvop.com/author/zyvop","authorSameAs":["https://zyvop.com","https://github.com/zyvop","https://x.com/zyvop1"],"category":"Scraping","tags":["python social media scraping tutorial 2026","scrape twitter x python 2026","scrape reddit python tutorial","instagram data scraping python","social media sentiment analysis python scraping"],"audience":"Developers, software engineers, and students learning Scraping","tone":"Practical and evidence-based engineering guidance","readingTimeMinutes":14,"wordCount":2997,"faqs":null,"primaryTopic":"Scraping","publishedAt":"2026-07-21T06:09:20.030Z","updatedAt":"2026-08-29T04:15:00.130Z","canonicalUrl":"https://api.zyvop.com/scraping-social-media-data-with-python-x-twitter-reddit-instagram-2026-k7og3"}