功能测试是否只是断言响应？

Question

I have some confused question about testing.

We're using Slim as framework of the system. As I know, Unit Test is the minimum of testing. For example, to test if the class or method works as expected.

To test another functions of the system, for example, the system provide an API to search products information, this search function designed by the following thoughts:

1. Clients post keyword to the API entry. eg. /search
2. Handle the request and inject SearchService into the controller.
3. Set the keyword to the SearhService.
4. Push the keyword into the database. eg. search_history.
5. Fetch the search result back to controller, and response to clients.

This is how search API designs.

And we created a functional test for the Search API called SearchTest. This is how we done:

1. Use $this->runApp() method provided by Slim to sending requests to the API entry.
2. Set several test cases to test different kind of scenario, such as "No keyword provided", "No search result" ... etc.
3. Assert the responses that return value or HTTP status as expected.

Some questions are confusing:

• Sending requests and asserting the responses, is that all? Do we misunderstand so called "functional testing"?
• We shouldn't care about how it works, we just need to send request and assert the responses, right?
• If not, should we check dependencies or used components work as expected? For example, should we connect to the database, and check if the keyword pushing success or not?

Answer 1

Automated Testing (and testing in general) is considered a good practice in Software Engineering. However, there exist a lot of hot discussion on what are the boundaries for test methodology and types classification.

In this sense, a practical approach to correctly implement a testing methodology is learning from those who you may consider respectable in your context of software development AND making sure that the practice you adopt plays nice for the goal you are willing to achieve. Try to be consistent on this.

Just as a reference, lets take this definition.

Functional testing refers to activities that verify a specific action or function of the code. These are usually found in the code requirements documentation, although some development methodologies work from use cases or user stories. Functional tests tend to answer the question of "can the user do this" or "does this particular feature work."

Considering this approach, you want your tests to give you confidence that the user is able to execute some function (or feature) provided by your application. It does not matter (here) how that feature is provided, just get into your user's shoes and think "Am I getting what I'm expecting from this action?", the answer to that question should hint you the assertions for your test.

Non-functional testing refers to aspects of the software that may not be related to a specific function or user action, such as scalability or other performance, behavior under certain constraints, or security. Testing will determine the breaking point, the point at which extremes of scalability or performance leads to unstable execution. Non-functional requirements tend to be those that reflect the quality of the product, particularly in the context of the suitability perspective of its users.

If you want to test how a certain function is executed, it's probably a Unit Test what may help you to do those assertions.

But again, remember it's not that of a sharp limit on what your tests should perform out from their name, but just to know what classification would fit best for the type of assertions you are performing (that helps you better structure your tests and give you a clear idea of what are you testing). Try to be consistent and focus on testing what you really need to be confident on.