In a recent project, an AI-powered chat application was successfully deployed from a local setup to Azure cloud infrastructure. The application, initially built using FastAPI, Docker, Postgres, Nginx, and llama.cpp, was adapted for cloud deployment to handle up to 1040 concurrent users with the current Azure credit limits. The cloud architecture leverages Azure Container Apps (ACA), which simplifies Kubernetes management, allowing for automatic scaling based on HTTP requests. The system uses Azure-managed PostgreSQL for database needs, ensuring scalability and reliability. Costs for this setup can range from $13 to $7000 monthly, depending on the hardware. The deployment process was streamlined using Azure CLI and infrastructure-as-code tools like Terraform. The app’s public URL was made memorable through DNS records, and future improvements include setting up CI/CD pipelines, enhancing monitoring, and optimizing the language model server for lower latency and higher concurrency.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
