Ultimate X (Twitter) Scraper

1. 👤 Twitter Account Information

The scraper collects detailed data for both User Profiles and Twitter Pages, as well as for accounts involved in tweets, replies, quotes, retweets, and reactions on the platform. Additionally, the same account information is gathered for all associated accounts, such as Followers, Verified Followers, Following, and Subscriptions.

Data Collected

1. Account Information: For the target profile or page, the following details are collected:

• Account Identifiers: Account ID, profile link, username, name, tagline, and verification status.

• Profile Info: Bio, bio links, location, categories, total tweet count, media count, and account joining date.

• Contact Info: Phone numbers, email addresses, and external links (such as websites or social profiles).

• Professional Info: Work-related information, pinned tweets, and subscription details.

• Profile Picture & Cover Photo: Downloads the account's profile picture and cover photo.

• Engagement Metrics: Follower count, following count, total tweet count, and media count.

2. Associated Accounts & Interactions: The scraper collects detailed information for associated accounts and interactions linked to the target profile or page, as outlined in the Account Information section above. This includes:

• Followers: Accounts that follow the target profile or page.

• Verified Followers: Accounts with blue verification checkmarks that follow the target profile.

• Following: Accounts the target profile follows.

• Subscriptions: Paid followers or subscriptions associated with the account.

3. Additional Account Types: The scraper also collects the same detailed account information for the following entities:

• Tweet Repliers: Accounts that reply to a tweet.

• Quoters: Accounts that quote a tweet.

• Retweeters: Accounts that retweet a tweet.

• Mentioned Accounts: Accounts mentioned in tweets or replies.

2. 📝 Twitter Tweets

Scrape all types of tweets from Twitter profiles, including:

• Account Posts: Tweets, retweets, and replies made by the account.

• Account Media Posts: Tweets containing media such as photos, videos, or GIFs.

• Account Highlights: Pinned or highlighted tweets by the account.

Data Collected

• Tweet Owner Details: Full account details, including username, profile link, bio, verification status, and all other attributes from the Account Information section.

• Tweet Content: Full tweet text (in any language), posting date and time, media links (photos, videos, GIFs), hashtags, and account mentions.

• Tweet Metadata: Tweet ID, tweet type (original, retweet, quote, reply), language, and whether the tweet contains sensitive content.

• Media: Any media in the post, including photos, videos, GIFs, all of which are downloadable.

• Engagement Metrics: See Engagement Data section for details on replies, quotes, retweets and likes.

3. 📊 Engagement Data (Replies, Quotes, Retweets & Likes)

The Engagement Metrics section applies to Twitter tweets, retweets, replies, and media, capturing detailed interactions for each post:

💬 Replies

• Reply Owner Details: Full account details of users who replied to the tweet, as mentioned in the Twitter Account Information section.

• Replies Count: Total number of replies for the tweet.

• Reply Metadata: Complete reply tweets data, as outlined in the Twitter Tweets section.

🗣️ Quotes

• Quoter Details: Full account details of users who quoted the tweet, as mentioned in the Twitter Account Information section.

• Quote Count: Total number of quotes on the tweet.

• Quote Metadata: Full quote tweets data, as outlined in the Twitter Tweets section.

🔄 Retweets

• Retweeter Details: Full account details of users who retweeted the post, as mentioned in the Twitter Account Information section.

• Retweet Count: Total number of retweets for the tweet.

❤️ Likes/Reactions

• Like Count: Total number of likes/reactions on the tweet.

4. 📈 Twitter Trends

The scraper captures the top daily Twitter trends for any specified location or category. It includes popular hashtags and keywords, categorized by topics such as news, sports, entertainment, and more.

Data Collected

• Trend Name: The name of the trending hashtag or keyword.

• Trend Rank: The ranking of the trend within the specified location.

• Total Tweets: The total number of tweets related to the trend.

• Tweets Data: Collects tweets information for each trend, as detailed in the Twitter Tweets section.

• Location: The region where the trend is active.

5. 🔍 Twitter Search Data & Advanced Filters

Our scraper allows you to comprehensively search and scrape tweets and user profiles on Twitter using keywords, hashtags, and a range of advanced filters. All or any combination of these filters can be applied to gather highly targeted data based on your specific criteria, providing flexibility in data collection.

Search Filters

• Keywords & Hashtags: 

  • Filters: Search for specific keywords, exact phrases, or hashtags within tweets or user profiles. Use this filter to find content that mentions specific topics or tags.

  • Example Usage: You can search for a specific brand name, product keyword, or campaign hashtag to gather all related tweets or users discussing that subject.

• Location: 

  • Filters: Filter tweets or profiles by geographic regions. This is useful for collecting data from specific locations or regions to focus on local discussions or events.

  • Example Usage: Find tweets from users in New York City or profiles of people located in a particular country.

• Date Range: 

  • Filters: Scrape tweets or profiles based on a specified time frame. Use this to capture content from a specific date range, such as during an event or a product launch.

  • Example Usage: Gather tweets from the last week or profiles that have been active in the past month.

• Mentions: 

  • Filters: Filter tweets or profiles that mention specific accounts. This helps you capture conversations around influencers, brands, or key people being discussed or tagged in tweets.

  • Example Usage: Scrape tweets where a brand account is mentioned or users who frequently mention a specific influencer.

• Replies: 

  • Filters: Choose to include or exclude tweets with replies or profiles that frequently engage in reply threads. This helps filter conversations based on engagement or participation.

  • Example Usage: Scrape reply threads to track customer feedback or remove tweets with replies to focus only on original posts.

• Links: 

  • Filters: Include or exclude tweets containing links or profiles that frequently share links. This filter is ideal for narrowing down posts with external content, such as blog posts or product links.

  • Example Usage: Focus on tweets that contain links to articles or product promotions or exclude them to focus on direct conversations.

• Minimum Engagement: 

  • Filters: Set minimum thresholds for replies, likes, or retweets in tweets or profiles. This filter helps you focus on the most engaging content, ensuring that you scrape only the most relevant or popular data.

  • Example Usage: Gather only tweets that have at least 100 retweets or profiles whose tweets regularly reach high engagement levels.

• Language:

  • Filters: Filter tweets or profiles by specific languages. This helps when you're targeting content in specific languages, ensuring the scraper only gathers relevant data.

  • Example Usage: Collect tweets written in English or profiles that post primarily in Spanish.

Data Collected

• Latest Posts: Scrapes the most recent tweets matching your search filters, capturing all relevant tweet attributes, including tweet text, media, engagement metrics and more as detailed in the Twitter Tweets section.

• Top Posts: Captures the most popular or engaging tweets based on metrics like likes, retweets, and comments, with full tweet data included as detailed in the Twitter Tweets section.

• People: Scrapes user profiles that match your keyword search and filters, collecting complete account details such as username, bio, profile picture,  follower count and more as outlined in the Twitter Account Information section.

📂 Sample Data 

The Features section provided an overview of the data collected by the scraper. For more in-depth information, you can explore the sample data files for each section. These files offer a comprehensive look at all the attributes and data points collected by the scraper: 👉 View Sample Data