The Tech Stack that Works for My (AI) Startups
In this post, I list the tech stack and related services that support our products at Takin.ai and HiddenArt.ai and the things we tried but did not work.
Update Log:
Nov. 2024:
- Switched from AWS to DigitalOcean for hosting: AWS is getting so complicated for simple hosting services we need.
- Switched from AWS SES to Postmark: I hope we did this earlier
- Switched from Weaviate to self-hosted PostgreSQL with pgvecor to lower cost and simplify our stack
Frontend
- Next.js/TypeScript hosted on Vercel:
- Pages Router (Old) vs. App Router (New): we used Pages router for one project and App Router for another project. It does not matter that much for the features I use so far - pick the one that your existing developers are most comfortable with and use a small project to push the team to learn the new features related to App Router
- tRPC: tried but did not use given my team is not familiar with this and I am not sure about the additional benefits
- TailwindCSS for CSS
- Hero Icons: SVG icons
- PlayWright for front-end testing
Note that we use Nextjs monorepo to include projects on API, Blog, Admin, Gallery, etc.
Backend
Now:
- DigitalOcean: hosting service switched from AWS.
- MongoDB: self-hosted NoSQL database.
- PostgreSQL: self-hosted for backend database.
- pgvector: PostgreSQL extension to support vector similarity search.
- Redis and Vercel KV: Redis database for storing chat conversation and Dify integration
- AWS S3 for cloud object storage
- Replicate: used as the backend models for most of text-to-image generation tools.
Previously:
- MongoDB Atlas for serverless database
- Qdrant: we used Qdrant to develop ChatPDF like features but switched to Weaviate for serverless - Qdrant has no serverless option and cannot be easily scaled down.
- Weaviate: our current vector database (Serverless Cloud Service)
- Redis and Vercel KV: Redis database for storing chat conversation and Dify integration
- ElasticSearch for searching functions
- FastAPI for Python backend
- AWS Fargate for hosting FastAPI backend in serverless
- OpenAI API for AI functions (both GPT 3.5 and 4)
- LangChain for jumpstarting some PoC projects but not used for many products due to the extra unnecessary complexity for our scenarios.
- Huggingface for data and model storage and some LLM integration such chatting using Llama 2
- AWS GPU: we started with AWS GPU G4 instances - way too expensive - bought some reserved instances to help but ended up using Runpod for cheaper price and easier setup.
- Runpod for Stable Diffusion backend hosting (great Stable Diffusion WebUI docker templates to choose from). We are in the process to switch to Runpod serverless.
- A1111 Stable Diffusion WebUI: our SD backend running in the API mode.
Code
Github for everything: code management and collaboration, issues, CI/CD
Data Collection
- Scrapy for Python Scraper. See my tutorial: A Minimalist End-to-End Scrapy Tutorial
- Zyte for hosting Scrapy spiders and avoiding banning.
Analytics
- Google Analytics for site analytics
- MongoDB Charts for basic BI dashboards
Payment
Stripe for everything related to payment
Dev Tools
CMS
Headless Wordpress hosted on SiteGround - see my other related post on Headless Wordpress + Next.js + ElasticSearch for more details.
Domain Names
I register all my domains mainly from the following websites - NameCheap is cheaper with additional features. The DNS servers are then changed to SiteGround to use company email service and CMS.
Now:
- Postmark for sending emails via API with templates
- Namecheap Business Email for company emails
Previously:
- AWS SES for programmatic email service
- SiteGround for free company emails, such as support@takin.ai - see my other related post on Setup Websites using SiteGround for more details.
Misc Tools
Hope this can provide some useful information for you.
PS. The featured image for this post is generated using HiddenArt tool from Takin.ai.