Empowering Developers with Meta Code Llama 70B Model

It is great to be working with Meta as they roll out the 70 billion parameter versions of their three Code Llama models to the open-source community. This is another significant step forward in extending the availability of cost-effective AI models to our Dell Technologies customers.

Code assistant large language models (LLMs) offer several benefits for code efficiency, such as enhanced code quality, increased productivity and support for complex codebases. Moreover, deploying an open-source LLM on-premises gives organizations full control over their data and ensures compliance with privacy regulations at the same time, reducing latency and controlling costs.

Meta has introduced their latest open-source code generation AI model built on Llama 2—the 70 billion parameter versions of the Code Llama models. Code Llama is a state-of-the-art LLM capable of generating code, and natural language about code, from both code and natural language prompts. Meta has shown that these new 70B models improve the quality of output produced when compared to the output from the smaller models of the series.

The Code Llama 70B models, listed below, are free for research and commercial use under the same license as Llama 2:

• Code Llama – 70B (pre-trained model)
• Code Llama – 70B – Python (pre-trained model specific for Python)
• Code Llama – 70B – Instruct (fine-tuned)

Dell PowerEdge XE9680 with NVIDIA H100 GPUs: A Powerhouse Solution for Generative AI

Dell Technologies continues its collaboration with Meta, providing the robust infrastructure required to support the deployment and utilization of these large language models. Dell Servers, such as the PowerEdge XE9680, an AI powerhouse equipped with eight NVIDIA H100 Tensor Core GPUs, are optimized to handle the computational demands of running large models such as Code Llama, while delivering maximum processing power, ensuring smooth and efficient execution of complex algorithms and tasks. Llama 2 is tested and verified on the Dell Validated Design for inferencing and model customization. With fully documented deployment and configuration guidance, organizations can get their generative AI (GenAI) infrastructure up and running quickly.

With Code Llama 70B models, developers now have access to tools that significantly enhance the quality of output, thereby driving productivity in professional software development. These advanced models excel in various tasks, including code generation, code completion, infilling, instruction-based code generation and debugging.

Use Cases

The Code Llama models offer a plethora of use cases that elevate software development, including:

• Code completion. Streamlining the coding process by suggesting code snippets and completing partially written code segments, enhancing efficiency and accuracy.
• Infilling. Addressing gaps in codebase quickly and efficiently, ensuring smooth execution of applications and minimizing development time.
• Instruction-based code generation. Simplifying the coding process by generating code directly from natural language instructions, reducing the barrier to entry for novice programmers and expediting development.
• Debugging. Identifying and resolving bugs in code by analyzing error messages and suggesting potential fixes based on contextual information, improving code quality and reliability.

As the partnership between Dell and Meta continues to evolve, the potential for innovation and advancement in professional software development is limitless. We are currently testing the new Code Llama 70B on Dell servers and look forward to publishing performance metrics, including the tokens per second, memory and power usage with comprehensive benchmarks in the coming weeks. These open-source models also present the opportunity for custom fine-tuning targeted to specific datasets and use cases. We are actively engaging with our customer community in exploring the possibilities of targeted fine-tuning of the Code Llama models.

Get Started with the Dell Accelerator Workshop for Generative AI

Dell Technologies offers guidance on GenAI target use cases, data management requirements, operational skills and processes. Our services experts work with your team to share our point of view on GenAI and help your team define the key opportunities, challenges and priorities.

Source: dell.com