Code-based testing is a crucial aspect of software development that ensures the integrity and quality of the code. It involves systematically testing the code to identify bugs, defects, and vulnerabilities before deploying the software. By implementing a robust testing plan, developers can minimize the chances of errors slipping into production code and enhance customer satisfaction.
What is Code Testing?
Code based testing involves a multitude of methodologies and techniques aimed at ensuring the reliability and quality of software. From unit testing to integration testing and beyond, developers have a range of strategies to choose from when validating their code.
A fundamental aspect of code testing is constructing a robust foundation of tests that cover diverse scenarios and edge cases. These tests act as a safety net, providing continuous feedback on the functionality and correctness of the code. By diligently testing their codebase, developers can identify and address issues early on, minimizing the time and effort spent on debugging and maintenance in the long run.
Furthermore, code testing plays a pivotal role in guaranteeing the overall quality and reliability of software. Thorough testing not only enhances customer satisfaction but also prevents potential issues that could lead to revenue loss or negative user experiences. By implementing a comprehensive testing plan, the software development process becomes more efficient, instilling confidence in both the development team and end users.
Code testing embraces various methodologies that go beyond any single approach.
:
Manual testing
involves human interaction with the system under test. Developers or end users manually test the code by performing various tasks, providing inputs, and verifying the outputs. This can be done by developers testing their own code or involving a sample of end users to test different functionalities and report any issues they encounter.
While manual testing is quick to start with, it has some drawbacks. Human testers are prone to errors, and for large-scale projects, it can be expensive to conduct extensive manual testing. However, manual testing provides the flexibility to thoroughly examine the software, and it can be effective in discovering usability issues and obtaining user feedback.
: To reduce costs and increase efficiency,
automated testing
uses scripts or tools to automate the testing process. Test scripts are created with predefined test cases and expected outcomes. These scripts simulate user interactions and verify the correctness of the software’s responses. In the event of a response deviating from the anticipated outcome, an error message or warning is triggered.
While creating automated test scripts requires more upfront time and resources, once established, they can be run multiple times throughout the software’s lifecycle. As the software evolves, the test scripts can be updated to accommodate new functionalities without the need for extensive manual retesting.
Also Read: Manual Testing vs Automation Testing
: Structured documentation is crucial in code-based testing to ensure clarity, facilitate understanding, and identify gaps in the testing process. Stakeholders, including non-technical individuals, may require insight into the testing procedures.
Documentation can take various forms, such as plain text files elucidating the program’s functionality, test objectives, or contextual comments embedded within the test code. The output produced by the test script should be well-written, allowing easy identification of errors and the specific areas where the program is not functioning as intended.
: Even if automated test suites pass all tests, it is important to account for potential regressions caused by changes in the code. Repeating the test script whenever a new feature is ready for deployment helps ensure that existing functionality is not inadvertently affected.
The terms code coverage and test coverage are relevant in testing. Code coverage refers to the percentage of code that is executed during testing, while test coverage measures the percentage of required features or specifications that are tested. Achieving 100% code coverage ensures that all code paths have been tested, reducing the chances of untested scenarios causing issues.
Also Read: Top 15 Code Coverage Tools
Code Testing Techniques
Coding testing techniques play a crucial role in ensuring the success of any software development project. These techniques involve a variety of testing approaches, spanning from evaluating small code components to assessing the overall functionality of the application. Let’s dive into the core five components of coding testing techniques:
:
Unit testing
is an integral part of the software development process. It involves testing small units or components of code individually to ensure their proper operation. This testing can be performed manually, but it is often automated in Agile and DevOps projects. Unit tests help identify issues in specific code units and ensure they function as intended.
:
Integration testing
focuses on combining individual software modules and testing them as a group. It occurs after unit testing and before functional testing. This testing stage verifies the interactions and compatibility between different modules, ensuring they work together seamlessly.
:
Functional testing
is conducted after integration testing. It involves testing the software to ensure that it meets all specified business requirements and functions correctly. The goal is to validate that the software has all the necessary features and capabilities for end users to utilize it without encountering any issues.
:
Regression testing
is performed to verify that software, which may have undergone changes such as enhancements, bug fixes, or compliance updates, still performs correctly. It ensures that the modifications made to the software do not introduce new issues or break existing functionality. Regression tests help maintain the overall quality and stability of the software across different releases.
Read More: How to run Regression Testing in Agile Teams
:
Acceptance testing
is carried out to evaluate the system’s acceptability from the end user’s perspective. The primary objective is to assess whether the software complies with the business requirements and is suitable for production deployment. Acceptance tests validate that the software meets the expectations and needs of the end users.
By incorporating these coding testing techniques into the software development lifecycle, teams can identify and rectify issues at different levels, ensuring a higher quality and more reliable software product.
How to perform Code Testing with Example
Code testing, such as unit testing, is a crucial practice for software developers. However, understanding how to effectively write unit tests for code can be challenging.
In this example on unit testing, the goal is to demonstrate that unit tests are actually quite straightforward. The real complications arise from poorly designed and untestable code. We will explore the factors that make code difficult to test and identify anti-patterns and bad practices to avoid, ultimately improving testability and reaping additional benefits. The aim is not only to make testing less burdensome but also to enhance the overall robustness and maintainability of the code itself.
Unit tests can focus on different behavioral aspects of the system under test, typically falling into two main categories: state-based and interaction-based testing. State-based unit testing involves verifying that the system produces correct results or that its resulting state is accurate. On the other hand, interaction-based unit testing involves checking whether certain methods are invoked correctly.
Read More: Best Practices for Unit Testing
To illustrate a proper software unit testing example, let’s imagine a mad scientist attempting to create a supernatural chimera with various animal parts. In this scenario, the scientist would need to ensure that each individual part works properly. Similarly, in unit testing, we follow the Arrange-Act-Assert steps. For instance, if we take a frog’s leg as a unit, we would stimulate it electrically and check if the muscle contraction is correct. The example provided demonstrates a simple unit test for a palindrome detector.
In the Arrange phase, the system under test (SUT) is created and set up. The Act phase involves invoking a method on the SUT and capturing the result or checking for expected side effects. Finally, the Assert phase confirms whether the method’s behavior aligns with expectations.
[TestMethod]
public void IsPalindrome_ForPalindromeString_ReturnsTrue()
{
//In the Arrange phase:
//Create and set up the system under test.
//The system under test can be a method, a single object, or a graph of //connected objects.
//It is acceptable to have an empty Arrange phase.
//For example, when testing a static method, the system under test //already exists in a static form, requiring no explicit initialization.
PalindromeDetector detector = new PalindromeDetector();
//In the Act phase:
//Invoke a method to interact with the system under test.
//Collect the returned result to verify its correctness.
//Check for expected side effects if the method doesn't return anything.
bool isPalindrome = detector.IsPalindrome("kayak");
//In the Assert phase:
//Validate the behavior of the method to determine the test's success or //failure.
//Compare the actual output with the expected outcome.
//Ensure that the method behaves consistently with the defined //expectations.
Assert.IsTrue(isPalindrome);
}By following these principles and techniques, developers can approach code testing, particularly unit testing, with confidence. Writing effective unit tests not only makes testing less complex but also contributes to the overall quality, robustness, and maintainability of the code.
Also Read: Top Unit Testing Frameworks in 2023
Must-have code testing tools
In the vast array of available testing tools and frameworks, selecting the most suitable ones for your specific needs can be a daunting task. To assist you in this endeavor, a curated list of code testing tools has been compiled to cater to various requirements. It is crucial to undertake a comprehensive exploration and evaluation of each tool to ensure its compatibility with your technology stack and alignment with your testing objectives. By conducting meticulous research, informed decisions can be made, enabling the establishment of effective and efficient testing practices.
Jasmine
: A behavior-driven development framework, independent of other JavaScript frameworks, with a clean syntax for testing JavaScript code.
Mocha
: A feature-rich JavaScript test framework that runs on Node.js and in browsers, providing flexible reporting and exception mapping.
Chai
: A powerful assertion library for Node.js and browsers, perfectly complementing any JavaScript testing framework.
QUnit
: A widely used JavaScript unit testing framework capable of testing any generic JavaScript code, employed by jQuery projects.
Sinon
: A standalone library offering test spies, stubs, and mocks for JavaScript, compatible with any unit testing framework.
Karma
: A test runner suitable for different browsers, providing detailed test results and aiding in cross-browser testing.
Selenium
: An automation tool primarily used for web application testing and web-based administrative tasks.
WebdriverIO
: Simplifies browser and mobile application control with concise code, managing the Selenium session efficiently.
Nightwatch
: A user-friendly Node.js-based testing solution for browser-based apps, utilizing the powerful W3C WebDriver API.
PhantomCSS
: Compares screenshots captured by Casper.js to baseline images, facilitating visual regression testing.
PhantomFlow
: Enables UI testing with decision trees, offering a unique approach based on code-described user flows.
Percy.io
: Delivers continuous visual integration by capturing DOM snapshots, providing iterative feedback for visual changes.
Remember to choose tools that align with your project requirements and empower you to write comprehensive tests that validate the functionality and reliability of your code.
Challenges in Code Testing
Here are some common challenges in code testing:
: Ensuring comprehensive test coverage across all code paths and scenarios can be challenging, especially in complex systems. Identifying and testing all possible combinations and edge cases can be time-consuming and resource-intensive.
Also Read: Test Coverage Techniques Every Tester Must Know
: Managing test data, including creating realistic and diverse data sets, can be a challenge. Test data needs to cover a wide range of scenarios, including valid and invalid inputs, boundary values, and various data types.
: Setting up and maintaining
test environments
that closely resemble the production environment can be difficult. Issues with configuration, dependencies, and compatibility between different components can impact the accuracy and reliability of test results.
: As code evolves and changes, test cases need to be updated and maintained. This can be challenging, especially when there are numerous test cases and frequent code changes. Ensuring test cases remain relevant and effective is crucial.
: In large-scale applications with intricate dependencies, testing becomes challenging. External systems, databases, APIs, and third-party services may introduce complexities that require special consideration and coordination for effective testing.
: Testing legacy code can be problematic due to outdated frameworks, lack of documentation, and tight coupling. Understanding and testing legacy systems with limited or no unit tests can be time-consuming and require specialized techniques.
: Implementing and maintaining
test automation frameworks
and tools can present challenges. Developing robust and maintainable automated test suites requires expertise in scripting, handling dynamic elements, and managing test data.
Also Read: Key Elements of an Effective Test Automation Strategy
: Limited timeframes and resources can hinder thorough testing. Prioritizing testing efforts, optimizing test execution, and making trade-offs become necessary to meet project deadlines.
: Identifying the root cause of failures and isolating issues in complex systems can be time-consuming. Troubleshooting and debugging require a deep understanding of the codebase and thorough analysis of test results.
: Integrating code testing into
continuous integration and deployment pipelines
can be challenging. Ensuring fast and reliable feedback loops, managing test environments, and coordinating with development teams require effective collaboration and tooling.
Remember that these challenges can vary depending on the specific project, technology stack, and organizational context. Addressing these challenges often requires a combination of technical expertise, collaboration, and adopting best practices in code testing.
Best Practices for Code Testing
Here are 10 impactful best practices for code testing:
Initiate testing from the early stages of development to identify and address issues promptly. Embrace shift-left testing to catch bugs early on and improve code quality.
Set precise testing objectives and prioritize critical functionalities. This focus ensures comprehensive testing of essential areas.
Create a well-structured testing plan, prioritizing activities based on risks and dependencies. Strategic planning optimizes resource allocation and
maximizes test coverage
.
Leverage automation tools like Selenium to streamline repetitive testing tasks. Automating tests saves time, improves efficiency, and enables faster feedback on code changes.
Practice writing test cases before coding to ensure code adheres to specifications. TDD serves as executable documentation and promotes code quality.
Practice isolated testing by separating code from external dependencies. Use mocking or stubbing techniques to simulate dependencies’ behavior.
Pay attention to boundary conditions and test edge cases rigorously. Thoroughly validate inputs, values, and scenarios at the limits of expected behavior.
Perform regular
regression testing
to ensure code changes or bug fixes don’t introduce new problems. Maintain a comprehensive suite of regression tests.
Encourage collaboration between developers and testers through open communication, knowledge sharing sessions, and code reviews. Collaborative efforts enhance test scenarios and align code implementation with testing efforts.
Integrate testing into your CI/CD pipeline for
continuous testing
. Automate tests as part of the build process. Continuous testing reduces defects and ensures thorough validation of code changes.
By following these impactful best practices and leveraging automation, collaboration, and continuous testing, you can elevate your code testing approach and deliver high-quality software products.
Boost your Agile testing with BrowserStack Automate and App Automate. Ensure code quality, cover all scenarios, streamline communication, and achieve exceptional results. Experience seamless testing on 3000+ browsers and real mobile devices. Elevate your testing game with BrowserStack for an unmatched user experience.
Try BrowserStack for Free
Closing Notes
Code testing is a critical practice that ensures the integrity and quality of software code. By implementing a robust testing plan, developers can identify and address issues early on, minimizing the chances of errors in production code. Thorough testing enhances customer satisfaction, prevents revenue loss, and instills confidence in both the development team and end users.
With the adoption of best practices, such as early testing, clear objectives, strategic planning, test automation, and collaboration, developers can optimize their code testing approach and deliver high-quality software products. Remember, code testing is an ongoing process that requires continuous improvement and adaptation to ensure software reliability and success.
Programming is writing code to solve problems. Software Engineering is the practice of using a structured process to solve problems. As engineers, we want to have a codebase we can change, extend, and refactor as required. Tests ensure our program works as intended and that changes to the codebase do not break existing functionality.
At my last job, I worked with a Senior Engineer to build out a microservices-based backend to replace our existing Django monolith. It was a greenfield project and we were encouraged to try new things. I was reading Python Testing with pytest and convinced the Senior Engineer to let me bring pytest into our project. This was fortuitous as it forced me to take the lead in writing the initial set of tests we used as a template for all of our services.
This experience reinforced the principles highlighted in The Pragmatic Programmer. It's about being pragmatic in what we test, how we test, and when we test; we should leverage tools and techniques that allow us to test our code as efficiently as possible. Testing needs to be easy and free of barriers; once testing feels like a chore, programmers won't do it... and this is how software quality slips.
We dread going into the code because either there are no tests or the tests that exist are so brittle that we're forced to rewrite tests as we write code. This is not what Software Engineering is about. Test should enable refactoring, not hamper our ability to make changes to the codebase. We should spend our time writing business logic, not wrestling with tests.
Testing is folklore in the sense that best practices and techniques are passed down from programmer to programmer while working on projects as part of a team. If you are new to the industry and are trying to grok testing, it's hard to figure out how to get started. It feels like there is a lot of conflicting advice out there, and that's because there is. Testing is opinionated, more-so than any other software engineering discipline. Folks are always arguing about what to test, how to test, and especially when to test.
This is the first in a series of posts that details my thought process for how I go about adding tests to a codebase. In this post, I provide a broad introduction to the world of testing so we can have a common vocabulary for future posts.
When we write code, we need to run it to ensure that it is doing what we expect it to. Tests are a contract with our code: given a value, we expect a certain result to be returned.
Running tests can be thought of as a feedback mechanism that informs us if our program works as intended:
While passing tests cannot prove the absence bugs, they do inform us that our code is working in the manner defined by the test. In contrast, a failing test indicates that something is not right. We need to understand why our test failed so we can modify code and/or tests, as required.
Tests give us confidence that our code is working as intended. A slower feedback loop hampers development as it takes us longer to find out if our change was correct. If our workflow is plagued by slow tests, we won't be running them as often. This will lead to problems down the line.
Tests should be deterministic, i.e. the same input will always result in the same output. If tests are non-deterministic, we have to find a way to account for random behavior inside of our tests.
While there is definitely non-deterministic code in production (i.e. Machine Learning and AI), we should try to make all our non-probabilistic code as deterministic as possible. There is no point of doing additional work unless our program requires it.
We can confirm our program works by running it. This could be manually running a command in the REPL or refreshing a webpage; in both cases, we are looking to see if our program does what it is supposed to do. While manual testing is fine for small projects, it becomes unmanageable as our project grows in complexity.
By automating our test suite, we can quickly verify our program works on-demand. Some developers even have their tests triggered to run on file save.
Let's go over some definitions so we have a common vocabulary going forward.
A System Under Test (SUT) is the entity that is currently being tested. This could be a line of code, a method, or an entire program.
Acceptance Criteria refers to the check we perform that allows us to accept output from the system. The specificity and range of acceptance criteria depends on what we are testing: medical device and aerospace require tests to be specific as there is a lot less room for error.
If Amazon makes a bad recommendation, it's not the end of the world. If IBM's Watson suggests the wrong surgery, it can be life threatening.
Testing refers to the process of entering Inputs into our System Under Test and validating Outputs against our Acceptance Criteria:
Hopefully the test failure provides enough contextual information for us to find out where to look.
A well-thought-out testing strategy paired with thorough test cases provides the following benefits:
If a program does anything of interest, it has interactions between functions, classes, and modules. This means a single line change can break our program in unexpected ways. Tests give us confidence in our code. By running our tests after we modify our code, we can confirm our changes did not break existing functionality as defined by our tests.
In contrast, modifying a code base without tests is a challenge. There is no way of knowing if things are working as intended. We are programming by the seat of our pants, which is quite a risky proposition.
Bugs cost money. How much depends on when you find them.
Fixing bugs gets more expensive the further you are in the Software Development Life Cycle (SDLC). True Cost of a Software Bug digs into this issue.
This one is a bit controversial, but I think writing code with tests in mind improves system design. A thorough test suite shows that the developer has actually thought about the problem in some depth. Writing tests forces you to use your own API; this hopefully results in a better interface.
All projects have time constraints and it's quite easy to get into the habit of taking shortcuts that increase coupling between modules leading to complex interdependencies. We have to be cognizant of solving problems with spaghetti code.
Knowing we have to test our code forces us to write modular code. If something is clunky to test, there might be a better interface we can implement. Taking the time to write tests forces mindfulness upon us; we take a deep breath before looking at the problem from the perspective of a user.
Once you write testable code by using patterns like dependency injection, you'll see how adding structure makes it easier to verify our code is doing what we expect it to.
Tests can be broadly classified into two broad categories: black box testing and white box testing.
Black Box Testing refers to testing techniques in which the tester cannot see the inner workings of the item being tested.
White Box Testing is the technique in which the tester can see the inner workings of the item being tested.
As developers, we perform white box testing. We wrote the code inside of the box and know how to test it thoroughly. This is not to say that there is not a need for black box testing, we should still have somebody perform testing at a higher level; proximity to the code can lead to blind spots in our tests.
The Automated Test Pyramid provides guidance on how to structure our testing strategy. It says we should write lots of fast and cheap unit tests and a small number of slow and expensive end-to-end tests.
The Test Pyramid is not a hard and fast rule, but it provides a good place to start thinking about a testing strategy. A good rule of thumb is to write as many tests at each level as you need to have confidence in your system. We should be writing tests as we write code, iterating towards a testing strategy that works for the project we are working on.
Unit tests are low-level tests that focus on testing a specific part of our system. They are cheap to write and fast to run. Test failures should provide enough contextual information to pinpoint the source of the error. These tests are typically written by developers during the Implementation phase of the Software Development Life Cycle (SDLC).
Unit tests should be independent and isolated; interacting with external components increases both the scope of our tests and the time it takes for tests to run. As we will see in a future post, replacing dependencies with test doubles results in deterministic tests that are quick to run.
How big should our unit test be? Like everything else in programming, it depends on what we are trying to do. Thinking in terms of a unit of behavior allows us to write tests around logical blocks of code.
The Test Pyramid recommends having a lot of unit tests in our test suite. These tests give us confidence that our program works as expected. Writing new code or modifying existing code might require us to rewrite some of our tests. This is standard practice, our test suite grows with our code base.
Try to be cognizant of our test suite growing in complexity. Remember, code that tests our production code is also production code. Take the time to refactor your tests to ensure they are efficient and effective.
Suppose we have the following function that takes a list of words and returns the most common word and the number of occurrences of that word:
def
find_top_word
(
words
)
# Return most common word & occurrences
word_counter
=
Counter
(
words
)
return
word_counter
.
most_common
(
1
)[
0
]
We can test this function by creating a list, running the find_top_word function over that list and comparing the results of the function to the value we expect:
def
test_find_top_word
():
words
=
[
"foo"
,
"bar"
,
"bat"
,
"baz"
,
"foo"
,
"baz"
,
"foo"
]
result
=
find_top_word
(
words
)
assert
result
[
0
]
==
"foo"
assert
result
[
1
]
==
3
If we ever wanted to change the implementation of find_top_words, we can do it without fear. Our test ensures that the functionality of find_top_word cannot change without causing a test failure.
Every complex application has internal and external components working together to do something interesting. In contrast to units tests which focus on individual components, integration tests combine various parts of the system and test them together as a group. Integration testing can also refer to testing at service boundaries of our application, i.e. when it goes out to the database, file system, or external API.
These tests are typically written by developers, but they don't have to be. By definition, integration tests are larger in scope and take longer to run than unit tests. This means that test failures require some investigation: we know that one of the components in our test is not working, but the failure's exact location needs to be found. This is in contrast to unit tests which are smaller in scope and indicate exactly where things have failed.
We should try to run integration tests in a production-like environment; this minimizes the chance that tests fail due to differences in configuration.
Suppose we have the following function that takes in a URL and a tuple of (word, occurrences). Our function creates a records and saves it to the database:
def
save_to_db
(
url
,
top_word
):
record
=
TopWord
()
record
.
url
=
url
record
.
word
=
top_word
[
0
]
record
.
num_occurrences
=
top_word
[
1
]
db
.
session
.
add
(
record
)
db
.
session
.
commit
()
return
record
We test this function by passing in known information; the function should save the information we entered into the database. Our test code pulls the newly saved record from the database and confirms its fields match the input we passed in.
def
test_save_to_db
():
url
=
"http://test_url.com"
most_common_word_details
=
(
"Python"
,
42
)
word
=
save_to_db
(
url
,
most_common_word_details
)
inserted_record
=
TopWord
.
query
.
get
(
word
.
id
)
assert
inserted_record
.
url
==
"http://test_url.com"
assert
inserted_record
.
word
==
"Python"
assert
inserted_record
.
num_occurrences
==
42
Notice how this is the kind of testing we do manually to confirm things are working as expected. Automating this test saves us from having to repeatedly check this functionality each time we make a change to the code.
End-to-end tests check to see if the system meets our defined business requirements. A common test is to trace a path through the system in the same manner a user would experience. For example, we can test a new user workflow: simulate creating an account, "clicking" the link in the activate email, logging-in for the first time, and interacting with our web application's tutorial modal pop-up.
We can conduct end-to-end tests through our user interface (UI) by leveraging a browser automation tool like Selenium. This creates a dependency between our UI and our tests, which makes our tests brittle: a change to the front-end requires us to change tests. This is not sustainable as either our front-end will become static or our tests will not be run.
A better solution is to test the subcutaneous layer, i.e. the layer just below our user interface. For a web application, this would be testing the REST API, both sending in JSON and getting JSON out.
Our subcutaneous tests are our contracts with our front-end; they can be used by our front-end developers as a specification of the REST API. Tools, like swagger-meqa, that are built on top of the OpenAPI Specification can help us automate this process. We could also full-featured tools like Postman to test, debug, and validate our API.
End-to-end tests are considered black box as we do not need to know anything about the implementation in order to conduct testing. This also means that test failures provide no indication of what went wrong; we would need to use logs to help us trace the error and diagnose system failure.
Here we are using the Flask Test client to run subcutaneous testing on our REST API. There are a lot of things happening behind the scene and the result we get back (HTTP status code) lets us know that the test either passed or failed.
def
test_end_to_end
():
client
=
app
.
test_client
()
body
=
{
"url"
:
"https://www.python.org"
}
response
=
client
.
post
(
"/top-word"
,
json
=
body
)
assert
response
.
status_code
==
HTTPStatus
.
OK
Each test case can be separated into the following phases:
There are two widely used frameworks for structuring tests: Arrange-Act-Assert and Given-When-Then.
The AAA pattern is abstraction for separating the different part of our tests:
def
test_find_top_word
():
# Arrange
words
=
[
"foo"
,
"bar"
,
"bat"
,
"baz"
,
"foo"
,
"baz"
,
"foo"
]
# Act
result
=
find_top_word
(
words
)
# Assert
assert
result
[
0
]
==
"foo"
assert
result
[
1
]
==
3
The clear separation between the phases allows us to see if our test method is trying to test too many different things at once. Arrange-Act-Assert is the pattern I use when writing tests.
GWT provides a useful abstraction for separating the different phases of our test:
GWT is widely used in Behavior Driven Development (BDD).
def
test_find_top_word
():
# Given a list of word
words
=
[
"foo"
,
"bar"
,
"bat"
,
"baz"
,
"foo"
,
"baz"
,
"foo"
]
# When we run the function over the list
result
=
find_top_word
(
words
)
# Then we should see `foo` occurring 3 times
assert
result
[
0
]
==
"foo"
assert
result
[
1
]
==
3
In order to prove that our program is correct, we have to test it against every conceivable combination of input values. This type of exhaustive testing is not practical so we need to employ testing strategies that allow us to select test cases where errors are most likely to error.
Seasoned developers can balance writing code to solve business problems with writing tests to ensure correctness and prevent regression. Finding this balance and knowing what to test can feel more like an art than a science. Fortunately, there are a few rules of thumb we can follow to make sure our testing is thorough.
We want to make sure that all relevant requirements have been implemented. Our test cases should be detailed enough to check business requirements. There is no point building something if doesn't it meet the criteria you set forth.
We have to test each statement at least once. If the statement has a conditional (if or while), we have to vary our testing to make sure we test all branches of the conditional. For example, if we have the following code:
if
x
>
18
:
# statement1
elif
18
>=
x
>=
35
:
# statement2
else
:
# statement3
To make sure we hit all branches of the above conditional, we need to write the following tests:
x < 1818 <= x <= 35x > 35Two test cases that result in the same output are said to be equivalent. We only require one of the test cases in order to cover that class of errors.
"There are 2 hard problems in Computer Science: cache invalidation, naming things, and off-by-1 errors."
This is one of the oldest jokes in programming, but there is a lot of truth behind it, we often confuse if we need a < or a <=. This is why we should always test the boundary conditions. Given the following example:
if
x
>
18
:
# statement1
else
:
# statement2
To ensure we thoroughly test the boundary conditions of the code snippet above, we would to have test cases for x=17, x=18, and x=19. Be aware that writing test cases becomes more complicated if our boundary has compound conditionals.
This is a great guide on testing boundary conditions.
This refers to any of the the following cases:
Focuses on tracing the control flow of the program with a focus on exploring the sequence of events related to the status of data objects. For example, we get an error if we try to access a variable that has been deleted. We can use Data Flow testing to come up with additional test cases for variables that have not be tested by other tests.
Past experience provides insights into parts of our code base that can lead to errors. Keeping a record of previous errors can improve the likelihood that you will not make that same mistake again in the future.
Figuring out what to test and doing it efficiently is what I mean when I say Art of Developer Testing. The only way to get better at testing is by writing tests, coming up come up better testing strategies, and learning about different testing techniques. Just like in software development, the more you know about something, the better you will become at it.
While there is a lot of interesting discussion about when to write tests, I feel it takes away from the point of testing. It doesn't matter when you write tests, it just matters that you write tests.
If you are interested in exploring this topic, I recommend the following links:
In this post, we got a broad introduction to the world of testing. Now that we are all on the same page, we can explore testing in more depth in future posts.