Introduction

In this exercise, you will build a small command-line interface application in Typescript, running on Node.js, that retrieves data from the public UNC CS Experience Labs’ Web API (Application Programming Interface). Your app will retrieve and present a list of student organization names and “slugs” (URL-safe abbreviations), as well as provide more detailed output about a spefic organization of interest. Making a request to a web API involves network input/output (IO) which is a relatively “slow” process. Thus, the network request will be made asynchronously and your solution will be written to handle the asynchrony.

In this exercise, you will gain exposure to new tools of the trade: linting, automatic source formatting, and testing. Additionally, you will rely on a 3rd party library, axiom, for handling the web requests (as opposed to fetch, as shown in class).

Setting up a complete suite of developer tools in a DevContainer requires some configuration and tinkering concerns beyond our scope at this point in the course, so we are going to provide a working sample container to begin your work in. Let’s go ahead and clone this repository before giving you a tour around and setting you off on your way!

0. Accept the GitHub Classroom Assignment

Accept Github Classroom Project via this link: https://classroom.github.com/a/G7-C73Sl

After accepting, you will be given a link to your repository. Follow it. Its URL convention is github.com/ followed by comp423-24f/ex01-async-io- and your Github Username.

Look for the green Code button and the Local tab. Copy your repository’s URL. Clone it to your machine wherever you’d like. At this point in your CS career, we are hopeful you are able to remember, or search to remind yourself, how to clone a git repository from GitHub. Searches along the lines of “clone git repository from github” should prove fruitful.

1. Open the cloned repository’s directory in a new VSCode Window

You will see the project files contain starter files. You will notice a .devcontainer directory already established.

2. DevContainer - Rebuild and Reopen in Container

Now for the magical step of opening your project in a DevContainer. Open VSCode’s command palette (Shortcut: Command+Shift+P on Mac and Control+Shift+P on Windows) and search for “Reopen in Container.” This step will may take a minute, but it should be faster than when you established your first dev container in class. Why? Because both are based on the same TypeScript image from Microsoft and you already have it downloaded!

3. After VSCode Reopens in the DevContainer’s Environment, Open a Terminal

You should be able to open a new terminal and see a prompt that looks like this: node -> /workspaces/$REPO (main), where $REPO is your GitHub repository’s name.

Congratulations, you have started up a new DevContainer! This bash shell is running inside the container, which has Node.js, TypeScript, and more installed. The /workspace is the working directory your terminal is in, which is where your repository’s files are mounted, and (main) refers to the branch your HEAD is on. This custom shell prompt is one of the niceties that was setup in the container build.

As part of the container setup, dependencies are installed via .devcontainer/devcontainer.json’s "postCreateCommand" running npm install. If this does not complete before components of the VS Code Editor load (such as the test plugin for Jest and Prettier) you may see some red areas in VSCode’s status bar down at its bottom edge. If this is the case, once the automatic startup script has completed, try reloading the window from the command palette.

Open a new Terminal in your DevContainer. Try the following two commands:

1. npm test - This runs the project’s unit tests. You should see 1 suite and 3 tests pass.
2. npm run build+start hello world - This builds the project and runs it. You should see the two CLI args hello and world printed in JSON list literal format.

If either of these commands failed, try running npm install from a terminal and reloading the complete VSCode Window upon completion. Triple check you are in the DevContainer.

4. Code Tour

Let’s walk through the starter files you are seeing in the repository at a high level. Most you will not need to touch, but you should understand the general purpose of each:

1. .devcontainer/devcontainer - A lot like the one you setup! However, if you open it you can see that additional customizations to VSCode have been established to automatically install some VSCode Extensions (Prettier for code formatting and Jest for unit tests) and some configuration options. Additionally, after the DevContainer is created we are installing 3rd party code in node_modules via npm install. Lastly, we run a git config command in order to add this project’s workspace as a safe directory.

2. node_modules - This directory contains all of our project’s 3rd party dependency code.

3. src/index.ts - This file boostraps execution into our project’s main function. You will notice it imports and calls main from src/main.ts and passes in the system variable process.argv. This variable contains the string arguments the command-line program was created with.

4. src/main.ts - This file serves as the entry point to our program. You will notice main is defined as async, though it does not yet make use of any await functionality. On the second line, you will notice a peculiar statement. It’s worth pausing to understand it! This is an example of destructuring assignment. The variable on the right-hand side is argv which is a string[]. Notice on the left-hand side it looks like we’re declaring a list literal, but what is actually happening is we are using destructuring assignment syntax. The first two strings in argv are the node binary and the script (e.g. dist/index.js), however our program only cares about the “arguments” that follow, so we are ignoring the first two values in argv and assigning the rest to a new string list named args.

5. test/main.spec.ts - This file contains unit tests in a behavior-driven style. Notice the significant use of arrow functions! Those are all anonymous function definitions! Notice this reads somewhat like a specification: “describe main: it should not log the program or script from argv”. This is one style of unit test possible in Jest. The value of this style is we can specify our requirements in English and then implement tests which verify they behave as expected. There is more to say about what is going on here, we will be back!

6. .gitignore - Directories ignored by our git repository because they contain built artifacts (e.g. dist and coverage), 3rd party code (node_modules), or Mac-specific hidden files for Finder (.DS_Store).

7. .prettierignore - Here, like with .gitignore, we tell Prettier, our automatic code reformatting library, to ignore files in built directories such as dist and node_modules. We only want prettier watching over our src and test directories!

8. .prettierrc - This file is for additional prettier configuration. Ours has a few rules you can hover over the names of for a descriptin of their impact in VSCode. Prettier will automatically reformat your source code on save to ensure everyone’s spacing standards, and other stylistic concerns, are always consistent. This frees your mind from the tedium of syntax style concerns and helps make everyone’s syntactical styles standard!

9. babel.config.json - Our test framework, Jest, requires Babel, which is a source-to-source transpiler that can transform (transpile) from certain dialects of JavaScript (e.g. TypeScript) into other dialects. Its usage in this project is beyond your concern, but know it’s just a utility needed for the project’s testing setup.

10. eslint.config.js - This file contains boilerplate for our code linter, ESLint. The ES is short for ECMA Script, which is the true formal name of the colloqial “JavaScript” programming language. Linters are popular tools for software engineering teams that help ensure everyone uses the same conventions and standards. For example, eslint is what will suggest you use const declarations for variables, instead of let, if your code never reassigns the variable later on.

11. jest.config.ts - Configuration for our unit testing framework Jest, created by developers at Facebook/Meta. Ours simply tells jest which directory to find our test files in: ./test.

12. package-lock.json - Opening this file you will see the many subdependencies needed by our top-level dependencies. In real projects in the Node.js ecosystem, you will quickly learn that lots of dependencies are common. This has its pros and cons, but generally it’s great for such a vibrant ecosystem of free, open source software to be available on a platform.

13. package.json - This file should look familiar. You will see that we have many new devDependencies related to getting TypeScript, Jest, Prettier, and ESLint all working properly (and together). You will see in scripts there are a number of additional subcommands for working with our project than: test (runs jest), coverage (runs test coverage report), build, start, build+start (builds TypeScript to JavaScipt and starts the program), clean (deletes the dist directory).

14. tsconfig.json - Configuration settings for TypeScript. Of note, you’ll see in types that both node and jest are added so that TypeScript is aware of the built-in globals of node and jest, along with their type information.

5. Requirements

The expected behavior of your program is as follows:

1. One of two subcommands is expected: list or get
2. If no subcommand is provided, an Error should be thrown with a helpful message. The message will be printed in the handler of index.ts
3. If a subcommand other than list or get is provided, an Error should be thrown with a helpful message, like expectation #2.
4. For the list subcommand, all organizations in the CSXL API should be printed. The output expected is one ($slug) $name per line where $slug is the organization’s slug and $name is its name. You can see the results of this API endpoint here: https://csxl.unc.edu/api/organizations.
5. For the get subcommand, an additional command-line argument with the slug of the organization is expected. If none is provided, an Error with a helpful message should be raised. If one is provided, you should use the API endpoint here: https://csxl.unc.edu/api/organizations/$slug replacing the last URL part with the slug. For example: https://csxl.unc.edu/api/organizations/unc-cs. The result of this subcommand is showing the name, description, website, and instagram of the student organization. Should a slug not exist and the API results in an error, your program should raise a custom Error with a helpful message.
6. Reach 100% test coverage with npm run coverage. Use a console spy for detecting output. Mock axios for HTTP requests. More on this in an upcoming section.

6. Starting with Command-Line Arguments

To help you get started, we will provide some guidance on cleaning up your main function and corresponding tests.

Go ahead and replace the logic in your program’s main function with the following:

export const main = async (argv: string[]) => {
  const [, , ...args] = argv;

  if (args.length === 0) {
    throw new Error("No subcommand provided. Expecting 'list' or 'get'.");
  } else {
    console.log(args);
  }
};

For an explanation of the const assignment line, see Code Tour part 4 above.

The args variable is a string[] that contains arguments given to the script following node and the index.js script. You can try building and running your program with the following examples to see the behavior. The lines prefixed with a $ should be run at your DevContainer’s shell prompt:

$ npm run build
$ npm start
No subcommand provided. Expecting 'list' or 'get'.
$ npm start list
[ 'list' ]
$ npm start get csxl
[ 'get', 'csxl' ]

Notice after builting, npm start, which runs your build program, is reasonably fast at testing out various command-line arguments. However, testing in this way is not automated. Let’s add some automated tests and demonstrate how code coverage with tests works.

Open test/main.spec.ts and add replace the existing tests with a single test for covering the throw branch:

import { main } from "../src/main";

describe("main", () => {
  it("should raise an error if called without args", async () => {
    await expect(() => main(["node", "script.js"])).rejects.toThrow();
  });
});

Consider this structure. We are establishing a test suite for main, which is imported from our source code. We add a single test describing an expected behavior. This style of test description is idiomatic because test cases read as “main should raise an error…”.

The expect function call is forming an assertion of sorts. This usage of expect is interesting because main is an async function, meaning it returns a Promise. When calling functions that return Promise values, the call is placed inside of an anonymous arrow function definition. Notice this is the very terse usage of arrow syntax and equivalent to writing () => { return main["node", "script.js"] }. Doing so allows jest to wait for the promise to resolve or reject and gives you a nice API for determining the result. In this case, we expect the Promise to reject because an Error is thrown. Hence the expect(...).reject.toThrow().

Try running this test with npm test, which, in turn, runs jest, our project’s unit testing framework. You should see output as follows:

 PASS  test/main.spec.ts
  main
    ✓ should raise an error if called without args (10 ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.339 s, estimated 2 s
Ran all test suites.

Introducing Test Coverage

Now, this tells you all of your tests passed, but it does not give you any assurance of your program being fully tested. Notice there is an entire branch which is untested: the else branch. Luckily, we have tooling to show us whether our code has full test coverage. Try the following command: npm run coverage. This runs jest with the --coverage option enabled. You should see output like follows:

> jest --coverage

 PASS  test/main.spec.ts
  main
    ✓ should raise an error if called without args (12 ms)

----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
----------|---------|----------|---------|---------|-------------------
All files |      80 |       50 |     100 |      80 |                   
 main.ts  |      80 |       50 |     100 |      80 | 11                
----------|---------|----------|---------|---------|-------------------
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.523 s, estimated 2 s
Ran all test suites.

Notice on branches, e.g. conditional if/else, the tool is telling you your tests cover 50% of branches and 80% of statements/lines (long lines can make single statements). In main.ts, notice it even tells you which line(s) are uncovered by your tests: line 11. This corresponds to the console.log(args) line.

Let’s try adding another test for coverage of this line. Note: you will pretty quickly remove this test, and the accompanying simple behavior, because it is not part of the final program, but following along with this step is instructive in learning how to write tests with coverage for console.log statements.

Add the second test, as shown below:

describe("main", () => {
  /** First test should still remain here... elided for write-up purposes. */

  it("should log the args if called with args", async () => {
    const logSpy = jest.spyOn(console, "log").mockImplementation(() => {});
    await main(["node", "script.js", "list"]);
    expect(logSpy).toHaveBeenCalledWith(["list"]);
  });
});

The line defining logSpy is quite fascinating. It’s watching, aka spying on the console object’s log method. This means it’s keeping track of calls to it and its arguments. Additionally, the chained method call is replacing the standard implementation of console.log with a mockImplementation that is a no-op: a function literal that does nothing. This prevents the built-in implementation from running and, thus, from sending output to the terminal when console.log is called in your code. Very cool! This is an important unit testing concept called mocking we will explore further.

Now when you run npm test, you should see results for both tests:

 PASS  test/main.spec.ts
  main
    ✓ should raise an error if called without args (15 ms)
    ✓ should log the args if called with args (2 ms)

Test Suites: 1 passed, 1 total
Tests:       2 passed, 2 total

Additionally, try checking coverage: npm run coverage:

----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
----------|---------|----------|---------|---------|-------------------
All files |     100 |      100 |     100 |     100 |                   
 main.ts  |     100 |      100 |     100 |     100 |                   
----------|---------|----------|---------|---------|-------------------

Woo! 100% coverage! Unfortunately, this program does not satisfy the requirements nor do your tests verify correctness of the requirements. In fact, the test you just added should not be kept because it goes against the expected behavior of your program. It was added only for this purpose of showing you how to test for output logged with console.log using a spy with mocking and to show you how test code coverage works.

Go ahead and make a git commit by adding all of the files you have changed to git’s stage and forming a new commit with a meaningful message of progress (such as “Completed section 6: test coverage”). Push your work to your repository on GitHub.

7. Using axios for Typed Web Requests

Rather than using the built-in fetch function, as shown in class, we will make use of a very popular (downloaded over 40 million times in the past week!) promise-based HTTP request library called axios. For more detail, see its npm page, home page, and git repository. It has two really nice features for us. First, it has a more ergonomic API than fetch and allows us to specify generic types to hint to TypeScript what the shape of the expected results are. Second, it is easy to mock in jest.

The axios library will be needed by our built application, not just in development, so we will save it as a true dependency. Run the following command in your DevContainer:

npm install --save axios

Notice this changes package.json and package-lock.json with metadata that explicitly establishes this dependency. It also installed the library in node_modules.

We will use only a small portion of the axios API, its get method for “getting” or downloading data from the web via a URL. The get method is generic and returns a promise whose data value is expected to implement the shape of the generic. That’s a lot of lingo, let’s give an example:

import axios from "axios";

interface Organization {
  slug: string;
  name: string;
  short_description: string;
  instagram: string;
  website: string;
}

/* Later in the program... in an async function definition like `main`... */

try {
  const result = await axios.get<Organization[]>("https://csxl.unc.edu/api/organizations");
  const organizations: Organization[] = result.data;
  console.log(organizations[0].name);
} catch {
  throw new Error("Error loading organizations list.")
}

Notice we established an interface for Organization that contains properties exposed by the API for each organization.

Then, in the try/catch, we await the result of axios.get() thanks to the Promise-based nature of axios’s implementation. The get method is generic and in the diamond syntax we specify that the expected shape of the response will be a list of organizations. If you open https://csxl.unc.edu/api/organizations in a browser, you will see this is the case. (PRO TIP: Install a JSON formatter plugin in your Google Chrome to make viewing API results easier. You can find a popular one named JSON Formatter here.)

The result object will have other HTTP response metadata we will learn about in due time. For now we only care about the data in the response. Notice, importantly, on the following line where the organizations variable is declared, we are able to explicitly type this variable as Organization[] and assign it result.data with type safety. This is thanks to the implementation of the generic get method of axios. The type specified in the diamond syntax informs the type of result.data.

It is worth noting the generic type specified in get is decided on trust that the API actually actually returns data in this shape. In the event the API and the generic type disagree, or the API implementation changes shape and returns something of a different type in the future, you will see surprising errors at runtime! This is a concern best addressed with integration testing, outside the scope of this project. For now, trust this API returns this shape (and more!) and will not change while you are working on the project.

You can use the same get method for integrating with the API endpoint that returns a single organization (shared in the requirements), but the generic type will be Organization.

8. Mocking axios Responses in jest Tests

Your implementation’s dependency on axios needs to be mocked in tests with two primary motivations:

1. We are focused on unit testing in this project, not integration testing. Unit testing requires isolating dependencies from the unit under test.
2. Mocking “expensive” operations with side effects, such as network input/output, helps ensure unit testing is always fast and predictable. If you are working on the project and don’t have internet access your tests will still run even if the program itself will not in that moment.

We are intentionally not providing any suggested structure to your project’s code beyond using the provided main function as your entrypoint. However, we do encourage you to take the opportunity to practice program design techniques for a well factored program, such as those you learned in COMP301. Strive to write a simple, maintainable, structured program.

We will provide the following structural hint toward jest test organization that mocks axios and both spies on and mocks console.log to be a no-op and establishes some test data:

import axios from "axios";
jest.mock("axios");

const organizations = [
  {
    slug: "unc-cs",
    name: "Computer Science Department",
    short_description: "The Computer Science Department at UNC Chapel Hill.",
    instagram: "https://www.instagram.com/unccompsci/",
    website: "https://cs.unc.edu/",
  },
  {
    slug: "csxl",
    name: "CS Experience Labs",
    short_description: "Student-centered labs for gaining experience in CS.",
    instagram: "unc",
    website: "https://csxl.unc.edu",
  },
];

describe("some unit of code under test (e.g. main)", () => {
  /* Mocks accessible via closure to before/after hooks and tests.*/
  const mockAxios = axios as jest.Mocked<typeof axios>;
  let logSpy: jest.SpyInstance;

  beforeEach(() => {
    /* Setup the console spy mocks. */
    logSpy = jest.spyOn(console, "log").mockImplementation(() => {});
  });

  afterEach(() => {
    /* Reset the state of all mocks. */
    jest.resetAllMocks();
  });

  it("should test something that depends on axios.get...", async () => {
    // Here we're mocking the resolved value of `get` where data is the organizations[] defined in globals 
    mockAxios.get.mockResolvedValue({ data: organizations });
    // TODO: Call unit of code that depended on axios.get<Organization[]>
    // TODO: Test return value of unit of code (or spied expectations)
  });

  it("should raise a nicer error when axios.get FAILS (Promise rejected)...", async () => {
    const axiosError = new Error("AXIOS ERROR");
    mockAxios.get.mockRejectedValue(axiosError);
    await expect(async () => { 
      /* call unit under test expecting `get` to raise a nicer, human error */
      /* if a function is async, be sure to await its call */
    }).rejects.not.toEqual(axiosError);
  });

});

From here you should be off and on your way implementing the requirements in Section #5 and testing all unit(s) in your program. Don’t forget to isolate dependencies of your units under test! Try to write a well factored program with simple unit(s)! Remember: the simpler your units of code, the simpler your unit testing job to verify them is.