Code coverage for Swift Package Manager based apps

Learn how to check utilizing SPM?

The Swift Package Manager permits you to create standalone Swift functions each on Linux and macOS. You possibly can construct and run these apps and you’ve got the flexibility to put in writing unit exams to your codebase. Xcode ships with the XCTest framework, however you could not know that that is an open supply library. It is accessible on each single platform the place you may set up Swift. This additionally signifies that you should utilize the very same assertion strategies from the framework that you just used to work with on iOS to unit check your SPM bundle. 📦

Let me present you the right way to make a model new mission utilizing the Swift Package deal Supervisor:

mkdir "myProject" && cd $_


swift bundle init


swift bundle init --type=executable

Each the library and the executable template incorporates a pattern check file with a dummy check case. You possibly can run exams in some ways, there may be built-in assist for parallel execution (you may even specify the variety of employees), you too can filter what to run by check goal, check case or you may consider only one check. ✅

swift check


swift check -l   #or `swift check --list-tests`


swift check --parallel


swift check --parallel --num-workers 2



swift check --filter myProjectTests.myProjectTests

The check result’s going to look considerably like this:

Check Suite 'All exams' began at 2020-01-16 16:58:23.584
Check Suite 'myProjectPackageTests.xctest' began at 2020-01-16 16:58:23.584
Check Suite 'myProjectTests' began at 2020-01-16 16:58:23.584
Check Case '-[myProjectTests.myProjectTests testExample]' began.
Check Case '-[myProjectTests.myProjectTests testExample]' handed (0.070 seconds).
Check Suite 'myProjectTests' handed at 2020-01-16 16:58:23.654.
     Executed 1 check, with 0 failures (0 sudden) in 0.070 (0.070) seconds
Check Suite 'myProjectPackageTests.xctest' handed at 2020-01-16 16:58:23.655.
     Executed 1 check, with 0 failures (0 sudden) in 0.070 (0.071) seconds
Check Suite 'All exams' handed at 2020-01-16 16:58:23.655.
     Executed 1 check, with 0 failures (0 sudden) in 0.070 (0.071) seconds

Processing check outcomes

If it’s essential to course of the result of the testing, that may be fairly difficult. I’ve created a small instrument that may convert your check outcomes right into a JSON file. It is known as Testify, you may seize it from GitHub. Let me present you the way it works:

swift check 2>&1 | testify
swift check --filter myProjectTests.myProjectTests 2>&1 | testify

Sadly, you may’t use the –parallel flag on this case, as a result of when you accomplish that you may solely get progress indication as a substitute of the ultimate check outcome output. Happily, you may nonetheless filter exams, so you do not have to attend for every little thing.

The swift check command returns the check outcomes on the usual error, as a substitute of the usual output. That is why it’s a must to redirect the stderr into the stdout by way of the 2>&1 flag.

If every little thing went nicely you may see a pleasant JSON output, similar to this one:

  "endDate" : 602416925.25200009,
  "kids" : [
    
      "endDate" : 602416925.25200009,
      "children" : [
        
          "endDate" : 602416925.25200009,
          "children" : [

          ],
          "startDate" : 602416925.19000006,
          "circumstances" : [
            
              "outcome" : "success",
              "className" : "myProjectTests",
              "moduleName" : "myProjectTests",
              "testName" : "testExample",
              "duration" : 0.062
            
          ],
          "sudden" : 0,
          "final result" : "success",
          "title" : "myProjectTests"
        
      ],
      "startDate" : 602416925.19000006,
      "circumstances" : [

      ],
      "sudden" : 0,
      "final result" : "success",
      "title" : "myProjectPackageTests.xctest"
    
  ],
  "startDate" : 602416925.19000006,
  "circumstances" : [

  ],
  "sudden" : 0,
  "final result" : "success",
  "title" : "Chosen exams"

Enabling code protection knowledge

Code protection is a measurement of what number of traces/blocks/arcs of your code are executed whereas the automated exams are operating.

I imagine that coverage studies are extraordinarily helpful for all the developer workforce. Venture managers can confer with the protection proportion if it involves software program high quality. The QA workforce may study protection studies & check all of the remaining elements or counsel new check concepts for the builders. Programmers can get rid of a lot of the bugs by writing correct unit / UI exams for the applying. A protection report helps them to analyse what must be performed as nicely. Xcode has a built-in protection report web page, however it’s a must to enable reports first. You possibly can obtain the very same factor with out utilizing Xcode, by merely offering an additional flag to the check command:

swift check --enable-code-coverage

Okay, that is effective, however the place is my report? 🤔

Learn how to show protection knowledge?

To this point so good, you might have generated the code protection report recordsdata, however they’re nonetheless in a extremely complicated file format. You want yet another extra instrument with the intention to show them correctly.

sudo apt-get set up llvm


brew set up llvm

echo 'export PATH="/usr/native/decide/llvm/bin:$PATH"' >> ~/.zshrc

echo 'export PATH="/usr/native/decide/llvm/bin:$PATH"' >> ~/.bashrc

Now you might be prepared to make use of llvm-cov which is a part of the LLVM infrastructure. You possibly can learn extra about it by operating man llvm-cov, however I will present you the right way to show some primary protection report for the pattern mission.

This command will generate the protection report to your exams, however provided that you’ve offered the --enable-code-coverage flag throughout testing. It is best to notice that these llvm-cov enter paths could range based mostly in your present system. In case you are utilizing Linux, it is best to merely give the xctest path as a parameter (e.g. .construct/x86_64-unknown-linux/debug/myProjectPackageTests.xctest on this case), the instrument profile is situated below the identical listing that is not a giant distinction, however nonetheless watch out with the platform title. Often you do not need to embrace the recordsdata out of your .construct & Checks listing, however you may specify your personal regex based mostly filter as nicely. 🔍

Placing every little thing collectively

You do not need to fiddle with these parameters, proper? Neither do I. That is why I made a helpful shell script that may determine every little thing based mostly on the present mission. Save your self a number of hours, right here is the ultimate snippet:

BIN_PATH="$(swift construct --show-bin-path)"
XCTEST_PATH="$(discover $BIN_PATH -name '*.xctest')"

COV_BIN=$XCTEST_PATH
if [[ "$OSTYPE" == "darwin"* ]]; then
    f="$(basename $XCTEST_PATH .xctest)"
    COV_BIN="$COV_BIN/Contents/MacOS/$f"
fi

llvm-cov report 
    "$COV_BIN" 
    -instr-profile=.construct/debug/codecov/default.profdata 
    -ignore-filename-regex=".construct|Checks" 
    -use-color

It is best to reserve it as cov.sh or one thing related. Add some permissions by utilizing chmod +x cov.sh and you might be able to run it by merely coming into ./cov.sh. Your protection report will seem like this:

After all when you run this script on a mission that has extra supply recordsdata & unit exams, it will produce a greater report. 😜

Conclusion

Utilizing check outcomes and protection knowledge is a pleasant strategy to present studies to different members in your workforce. By operating these instructions on a steady integration server (like Bitrise), you may automate your total workflow.