akash

Prop 253: Streamline Open-Source LLMs Deployment from HuggingFace onto Akash

This proposal seeks 9,615 AKT, equivalent to $50k, calculated at the rate of $5.20 per AKT at the time of submission, from Akash Network for developing and open-sourcing a tool aimed at streamlining the deployment of LLMs onto Akash, thereby increasing Akash Network adoption. The system is capable of providing solutions for 95%+ text-text (txt-txt) and text-image (txt-image) LLMs available on HuggingFace. It enables an instant deployment of any existing or upcoming LLM from Hugging Face onto the Akash Network (supported by HuggingFace transformers, vLLM, GGML). This tool streamlines the deployment process of LLMs and automatically creates an inference API endpoint, saving developers time and resources, attracting new users to Akash.

Problem

Currently, deploying LLMs on Akash can be a complex and time-consuming process, requiring extensive technical expertise and configuration. This discourages developers, hindering LLM adoption on Akash and limiting the network's potential.

Solution

Our solution solves this challenge by offering a user-friendly, one-click deployment system. Developers can seamlessly deploy any LLM available on HuggingFace (supported by HuggingFace transformers, vLLM, GGML) onto Akash without manual configuration or coding. This significantly reduces deployment time and opens up Akash to a wider developer audience, enhancing adoption and ultimately the growth of Akash network.

Benefits

For Developers:

  • Save time and effort with instant, one-click deployments of LLMs onto Akash network.
  • No need to tackle the complex challenges of creating Docker images for specific large language models (LLMs). Users can simply launch our solution and choose whichever LLM they want to deploy on Akash with a single click.
  • Access diverse LLMs by directly downloading from HuggingFace and deploying them onto Akash with a user-friendly interface.
  • Leverage Akash's low-cost, scalable cloud infrastructure.

For Akash Network:

  • Attract new developers and boost LLM adoption within Akash.
  • Enhance user experience and increase platform usage.
  • Strengthen Akash's position as a leading platform for LLM execution.

Work Breakdown

Phase 1: Plan & Develop (Budget: $40,000)

Universal Backend (Multiple LLMs in a single Deployment) (140 hours):

  • Implement proper configuration to load all the LLMs available on HuggingFace supported by transformers.
  • Design and develop backend UI to simplify the management of LLMs for the deployer.
  • Proper Devops/Docker-Image configuration to deploy on Akash through cloudmos.
  • Enhance security and scalability of the core deployment engine.
  • Implement clear deployment controls and progress indicators.
  • Implement robust error handling and logging capabilities, Support for future LLMs that are yet to be launched.

API & PASSWORD Configurations (90 hours):

  • Implement robust API rate limiting and error handling.
  • Secure API configuration with a password to enable deployers to restrict/Limit access to their backend API calls.
  • Integrate user-specific configuration options.

Frontend User Interface (70 hours):

  • Design and develop a user-friendly, intuitive web interface.
  • Implement a configuration that enables deployers to add API, PASSWORD, and LLM MODEL IDs that they launched in the backend.
  • Proper dropdown to switch b/w working LLMs added by the user.

Dedicated LLM deployment (Backend + FE) (100 hours):

  • Provide an option to create dedicated deployments ‘Deploy on Akash’ for specific LLMs by establishing a deployment pipeline with Cloudmos.
  • Enhance security and scalability of the core deployment engine.
  • Implement robust error handling and logging capabilities, Support for future LLMs that are yet to be launched.

Phase 2: Open-Sourcing & Community Building (Budget: $10,000)

Documentation & Tutorials (50 hours):

  • Create comprehensive documentation covering installation, usage, and customization.
  • Develop detailed tutorials for common deployment scenarios.
  • Translate documentation into key developer languages.

Open-Source Release & Community Support (30 hours):

  • Open-source the project on GitHub under an appropriate license.
  • Establish a forum for community support and discussion under GitHub issues.
  • Respond to user queries and contribute to bug fixes.

Marketing & Outreach (20 hours):

  • Promote the project through relevant developer communities and publications.
  • Engage with LLM and Akash developers to gather feedback and build momentum.

Timeline

  • Phase 1: 3 months
  • Phase 2: 2 months

Budget Justification

The requested budget of $50,000 covers the development, open-sourcing, and community-building efforts outlined above. $100/hr Hourly rates are based on market averages for qualified developers and reflect the expertise required for this project (AI/ML, UI/UX, Web (FE + BACKEND), Devops, Cloud, Docker, Python & Security). We believe this investment is justified by the significant value proposition this tool offers to developers, Akash Network Growth, and Community.

Proposer

  • @HoomanHQ & Team:

    • Demonstrated a robust track record of contributions in the Akash ecosystem.
    • Recent notable contribution includes the complete development of the Akash Network website, accessible at the official link.

  • Advisor:

    • Alani Kuye (Former core-team member of 'Akash Network' / 'Overclock Labs')

Call to Action

We believe this project has the potential to significantly improve LLM accessibility and adoption within the Akash Network. We invite Akash Network Community to partner with us by providing the requested funding. We are confident that this collaboration will benefit all stakeholders and further propel Akash towards its vision of a democratized cloud computing future.

Thank you

and github discussion link: Streamline Open-Source LLMs Deployment from HuggingFace onto Akash