Get Tweet Data for Analysis
Learn how to obtain Twitter/X data for sentiment analysis and threat detection research
Access real-time tweet data using the official X (Twitter) Developer API. This is the most reliable method for obtaining live tweet streams for analysis.
Steps:
- Sign up for X Developer Account at developer.twitter.com
- Create a new app and generate API credentials
- Choose your access tier (Basic: $100/month, Pro: $5,000/month)
- Use the API to search tweets, get user timelines, or stream real-time data
- Export data in JSON or CSV format for analysis
Clone the TweetPhish AI repository and run it locally. Upload your own tweet datasets in JSON or CSV format for sentiment analysis and threat detection.
Quick Start:
# Clone the repository
git clone https://github.com/nilesh1013/tweetphish-ai.git
# Install dependencies
cd tweetphish-ai
pnpm install
# Set up database and run
pnpm db:push
pnpm devUpload Your Dataset:
Place your tweet dataset (JSON/CSV) in the project directory and run the import script:
pnpm exec tsx scripts/import-large.mjsAccess publicly available Twitter datasets from research repositories and academic sources.
Kaggle Datasets
Twitter spam, phishing, and sentiment analysis datasets
GitHub Datasets
Community-contributed tweet collections and research data
This project is based on published research on real-time phishing detection in tweets. Read the full paper for methodology and technical details.
Questions or need help? Contact me at [email protected]