Skip to main content

Statement testing with MS Copilot

Task 1 (student 1)

Using MS Copilot  identify statements for the insertSort function. Assign a tag to for each identified statement.  Present the list of statements with tags. Write the prompt and the chatbot answer. Discuss the result.

Does it properly recognized that the FOR instruction consists of 3 statements? 

Task 2 (student 2)

Using MS Copilot generate the Control Flow Graph in the DOT language for the insertSort function. Do not force Copilot to create the instruction list. Write the prompt and the chatbot answer. Discuss the result. 

Does the CFG has the same structure as the CFG created by a human in the previous section of the book?

When the Task 1 is finished, use the generated instruction list with tags as the hint for Copilot. Did it help them co correct the CFG (now the CFG should be correct)? Discuss the result.

Prompt: Generate for me Control Flow Graph in the DOT language for the insertSort function from wikipedia.

Answer: Certainly! Below is the Control Flow Graph (CFG) of the insertSort function in the DOT language:

digraph InsertSortCFG {
    entry [label="Entry"];
    exit [label="Exit"];

    entry -> init_i;
    init_i -> cond_i [label="i = 1"];
    cond_i -> exit [label="i >= length"];
    cond_i -> init_value [label="i < length"];

    init_value -> init_j [label="value = a[i]"];
    init_j -> cond_j [label="j = i - 1"];
    cond_j -> assign_a_jp1 [label="a[j + 1] = value"];
    cond_j -> body_j [label="j >= 0 && a[j] > value"];

    body_j -> shift [label="a[j + 1] = a[j]"];
    shift -> update_j;
    update_j -> cond_j [label="j--"];
    
    assign_a_jp1 -> update_i;
    update_i -> cond_i [label="i++"];
}

image.png

Task 3. (student 3)

Convince Copilot to write an instruction test for the function insertSort in the C language. Write the prompt and the chatbot answer. Discuss the result, compare it with the code written by a human in the previous section of the book.

Prompt: 

write an instruction test for the function insertSort in the C language. Here is the function:
void insertSort(int a[], int length)
{
int i, j, value;
for (i = 1; i < length; ++i)
{
value = a[i];
for (j = i - 1; j >= 0 && a[j] > value; --j)
{
a[j + 1] = a[j];
}
a[j + 1] = value;
}
}
Keep in mind to precisely test various data types and values, compare them with expected results

Answer:

Sure! Here's an instruction test for the insertSort function in C. This test will cover various data types and values, and compare them with the expected results.

#include <stdio.h>
#include <assert.h>


// Function prototype
void insertSort(int a[], int length);


// Function to print array (for debugging purposes)
void printArray(int a[], int length) {
    for (int i = 0; i < length; i++) {
        printf("%d ", a[i]);
    }
    printf("\n");
}


// Test function
void testInsertSort() {
    // Test case 1: Normal case
    int arr1[] = {5, 2, 9, 1, 5, 6};
    int expected1[] = {1, 2, 5, 5, 6, 9};
    insertSort(arr1, 6);
    for (int i = 0; i < 6; i++) {
        assert(arr1[i] == expected1[i]);
    }


    // Test case 2: Already sorted array
    int arr2[] = {1, 2, 3, 4, 5};
    int expected2[] = {1, 2, 3, 4, 5};
    insertSort(arr2, 5);
    for (int i = 0; i < 5; i++) {
        assert(arr2[i] == expected2[i]);
    }


    // Test case 3: Array with negative numbers
    int arr3[] = {-3, -1, -2, -5, -4};
    int expected3[] = {-5, -4, -3, -2, -1};
    insertSort(arr3, 5);
    for (int i = 0; i < 5; i++) {
        assert(arr3[i] == expected3[i]);
    }


    // Test case 4: Array with duplicates
    int arr4[] = {4, 2, 2, 8, 3, 3, 1};
    int expected4[] = {1, 2, 2, 3, 3, 4, 8};
    insertSort(arr4, 7);
    for (int i = 0; i < 7; i++) {
        assert(arr4[i] == expected4[i]);
    }


    // Test case 5: Single element array
    int arr5[] = {1};
    int expected5[] = {1};
    insertSort(arr5, 1);
    assert(arr5[0] == expected5[0]);


    // Test case 6: Empty array
    int arr6[] = {};
    int expected6[] = {};
    insertSort(arr6, 0);
    // No assertion needed for empty array


    printf("All test cases passed!\n");
}


int main() {
    testInsertSort();
    return 0;
}


This test function covers various scenarios:

  1. A normal case with mixed values.
  2. An already sorted array.
  3. An array with negative numbers.
  4. An array with duplicate values.
  5. A single element array.
  6. An empty array.

Each test case uses assert to compare the sorted array with the expected result. If all assertions pass, it prints "All test cases passed!" to indicate success.

 

Summary

Copilot created seperate function for printing array elements, which is good as it provides good code organization. Whole testing takes place in function testInsertSort(). There are 6 cases, including common numbers array, negative numbers array, empty list,array, sorted list,array, single element list and array with duplicates.

Running the code revealed an important issue with function:

main.c: In function ‘testInsertSort’:
main.c:77:5: warning: ‘insertSort’ accessing 4 bytes in a region of size 0 [-Wstringop-overflow=]
   77 |     insertSort(arr6, 0);
      |     ^~~~~~~~~~~~~~~~~~~
main.c:77:5: note: referencing argument 1 of type ‘int[0]’
main.c:11:6: note: in a call to function ‘insertSort’
   11 | void insertSort(int a[], int length)
      |      ^~~~~~~~~~
All test cases passed!

It turns out, that when empty array is passed, there is an overflow, as j value is not initialazed it defaults to 0. Then it tries to reach a[1] element of array, which triggers overflow warning. Even though, the code still runs and returns 0.

Back to top