Play with Code Llama 70B

Simi included in AI

2024-01-31 926 words 5 minutes

/en/play-with-code-llama/featured-image.png

Contents

Meta launches Code Llama in August 2023, free for research and commercial use.

A Code Llama 70B version was released in late January 2024, capable of generating and debugging programming strings significantly larger than previous models, meaning developers could give it more context about what they were coding, and it would understand more accurately.

Code Llama 70B scored 53 percent in accuracy on the HumanEval benchmark, performing better than GPT-3.5’s 48.1 percent and closer to the 67 percent mark an OpenAI paper reported for GPT-4.

There are three versions of Code Llama 70B:

Code Llama 70B: The base code model;
Code Llama 70B Python: Specialized for Python;
Code Llama 70B Instruct: Fine-tuned for understanding natural language instructions..

Code Llama 70B can be deployed locally via Ollama:

1

ollama run codellama:70b

Instruct

The instruct model was trained to output human-like answers to questions. This model variation is the easiest to use and will behave closest to ChatGPT, with answer questions including both natural language and code:

Prompt

1

ollama run codellama:70b-instruct 'You are an expert programmer that writes simple, concise code and explanations. Write a python function to generate the nth fibonacci number.'

Response

The Fibonacci sequence is a series of numbers in which each number is the sum of the two preceding numbers. The first two terms are 0 and 1, making the sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34…

Here’s a python function that generates the nth Fibonacci number using recursion:
1
2
3
4
5
6
7
8
def fibonacci(n):
   if n < 2:
       return n
   else:
       return fibonacci(n-1) + fibonacci(n-2)

# Example usage:
print(fibonacci(5)) # Outputs 5, the fifth Fibonacci number.

Code Review

Code reviews can often be time consuming for teams. Code Llama can help by providing a second set of eyes on a commit or pull request for simple bugs:

Prompt

1
2
3
4
5
6
7
8
9


ollama run codellama:70b-instruct '
Where is the bug in this code?

def fib(n):
    if n <= 0:
        return n
    else:
        return fib(n-1) + fib(n-2)
'

Response

The bug in this code is that it doesn’t handle the case when n is less than or equal to zero. The function should return 0 when n is less than or equal to zero, not n itself. As a result, negative numbers will cause the function to run infinitely and eventually overflow.

Here is the corrected version of the code:
1
2
3
4
5
def fib(n):
    if n < 2:
        return 1
    else:
        return fib(n-1) + fib(n-2)

Unit Tests

Writing unit tests often requires quite a bit of boilerplate code. Code Llama can help:

Prompt

1

ollama run codellama:70b-instruct "write a unit test for this function: $(cat fib.py)"

Response

Here’s a possible unit test for this function using the Python unittest library:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import unittest

class TestFibonacci(unittest.TestCase):
    def test_fibonacci(self, fibonacci):
        self.assertEqual(fibonacci(0), 0)
        self.assertEqual(fibonacci(1), 1)
        self.assertEqual(fibonacci(2), 1)
        self.assertEqual(fibonacci(3), 2)
        self.assertEqual(fibonacci(4), 3)
        self.assertEqual(fibonacci(5), 5)
if __name__ == '__main__':
   unittest.main()
This test verifies that the fibonacci function correctly calculates and returns Fibonacci numbers up to n=5. It uses assertions to check if the expected value matches the actual output of the function for each input.

Code Completion

The standard code completion model will simply continue to generate subsequent tokens based on the provided prompt. This variation works best if you’re only looking to generate code, such as having the model write a function after seeing the function signature or comment:

Prompt

1

ollama run codellama:70b-code '# A simple python function to remove whitespace from a string:'

Response

1
2
3
4
5


def remove_whitespace(string):
    return "".join(string.split())

# An example of using this function:
print(remove_whitespace("hello world"))  # -> helloworld

Infill

Fill-in-the-middle (FIM), or more briefly, infill is a special prompt format supported by the code completion model can complete code between two already written code blocks. This is especially effective for inserting code within an existing function such as below:

1
2
3
4
5


def compute_gcd(x, y):

    <FILL>

    return result

With Code Llama, infill prompts require a special format that the model expects.

1

<PRE> {prefix} <SUF>{suffix} <MID>

To use this with existing code, split the code before and after in the example above the into parts: the prefix, and the suffix. For example, for our LCM example above:

Prompt

1

ollama run codellama:70b-code '<PRE> def compute_gcd(x, y): <SUF>return result <MID>'

While results will vary, you should get something like this:

Response

1
2
3
4
5
6
7
8
9


if x == y:
        return x

    if x > y:
        x = x - y
    else:
        y = y - x

    result = compute_gcd(x, y)

Note: the model may return <EOT> at the end of the result. This is a special token in the response that represents the end of the response similar to <PRE>, <SUF> and <MID>

Python Model

Prompt

1
2
3


ollama run codellama:70b-python '
# django view for rendering the current day and time without a template
def current_datetime(request):'

Response

now = datetime.now()

html = “It is now %s.” % now

return HttpResponse(html)

Tools Built on Code Llama

Cody has an experimental version that uses Code Llama with infill support.
Continue supports Code Llama as a drop-in replacement for GPT-4
Fine-tuned versions of Code Llama from the Phind and WizardLM teams
Open interpreter can use Code Llama to generate functions that are then run locally in the terminal

Contents

Play with Code Llama 70B

Instruct

Code Review

Unit Tests

Code Completion

Infill

Python Model

Tools Built on Code Llama

References