Skip to main content

Speeding up Test Execution, Spring Context Mocking

· 3 min read
Haril Song
Owner, Software Engineer at 42dot

Overview

Writing test code in every project has become a common practice. As projects grow, the number of tests inevitably increases, leading to longer overall test execution times. Particularly in projects based on the Spring framework, test execution can significantly slow down due to the loading of Spring Bean contexts. This article introduces methods to address this issue.

Write All Tests as Unit Tests

Tests need to be fast. The faster they are, the more frequently they can be run without hesitation. If running all tests once takes 10 minutes, it means feedback will only come after 10 minutes.

To achieve faster tests in Spring, it is essential to avoid using @SpringBootTest. Loading all Beans causes the time to load necessary Beans to be overwhelmingly longer than executing the code for testing business logic.

@SpringBootTest
class SpringApplicationTest {

@Test
void main() {
}
}

The above code is a basic test code for running a Spring application. All Beans configured by @SpringBootTest are loaded. How can we inject only the necessary Beans for testing?

Utilizing Annotations or Mockito

By using specific annotations, only the necessary Beans for related tests are automatically loaded. This way, instead of loading all Beans through Context loading, only the truly needed Beans are loaded, minimizing test execution time.

Let's briefly look at a few annotations.

  • @WebMvcTest: Loads only Web MVC related Beans.
  • @WebFluxTest: Loads only Web Flux related Beans. Allows the use of WebTestClient.
  • @DataJpaTest: Loads only JPA repository related Beans.
  • @WithMockUser: When using Spring Security, creates a fake User, skipping unnecessary authentication processes.

Additionally, by using Mockito, complex dependencies can be easily resolved to write tests. By appropriately utilizing these two concepts, most unit tests are not overly difficult.

warning

If excessive mocking is required, there is a high possibility that the dependency design is flawed. Be cautious not to overuse mocking.

What about SpringApplication?

For SpringApplication to run, SpringApplication.run() must be executed. Instead of inefficiently loading all Spring contexts to verify the execution of this method, we can mock the SpringApplication where context loading occurs and verify only if run() is called without using @SpringBootTest.

class DemoApplicationTests {  

@Test
void main() {
try (MockedStatic<SpringApplication> springApplication = mockStatic(SpringApplication.class)) {
when(SpringApplication.run(DemoApplication.class)).thenReturn(null);

DemoApplication.main(new String[]{});

springApplication.verify(
() -> SpringApplication.run(DemoApplication.class), only()
);
}
}
}

Conclusion

In Robert C. Martin's Clean Code, Chapter 9 discusses the 'FIRST principle'.

Reflecting on the first letter, F, for Fast, as mentioned in this article, we briefly introduced considerations on speed. Once again, emphasizing the importance of fast tests, we conclude with the quote:

Tests must be fast enough. - Robert C. Martin

Reference

Fixture Monkey 0.4.x

· 3 min read
Haril Song
Owner, Software Engineer at 42dot
warning

As of May 2024, this post is no longer valid. Instead, please refer to Making Testing Easy and Convenient with Fixture Monkey.

Overview

With the update to FixtureMonkey version 0.4.x, there have been significant changes in functionality. It has only been a month since the previous post1, and there have been many modifications (ㅠ) which was a bit overwhelming, but taking comfort in the active community, I am writing a new post reflecting the updated features.

Changes

LabMonkey

An experimental feature has been added as an instance.

LabMonkey inherits from FixtureMonkey and supports existing features while adding several new methods. The overall usage is similar, so it seems that using LabMonkey instead of FixtureMonkey would be appropriate. It is said that after version 1.0.0, the functionality of LabMonkey will be deprecated, and the same features will be provided by FixtureMonkey.

private final LabMonkey fixture = LabMonkey.create();

Change in Object Creation Method

The responsibility has shifted from ArbitraryGenerator to ArbitraryIntrospector.

Record Support

Now, you can also create Record through FixtureMonkey.

public record LottoRecord(int number) {}
class LottoRecordTest {

private final LabMonkey fixture = LabMonkey.labMonkeyBuilder()
.objectIntrospector(ConstructorPropertiesArbitraryIntrospector.INSTANCE)
.build();

@Test
void shouldBetween1to45() {
LottoRecord lottoRecord = fixture.giveMeOne(LottoRecord.class);

System.out.println("lottoRecord: " + lottoRecord);

assertThat(lottoRecord).isNotNull();
}
}
lottoRecord: LottoRecord[number=-6]

By using ConstructorPropertiesArbitraryIntrospector to create objects, you can create Record objects. In version 0.3.x, the ConstructorProperties annotation was required, but now you don't need to make changes to the production code, which is quite a significant change.

In addition, various Introspectors exist to support object creation in a way that matches the shape of the object.

Plugin

private final LabMonkey fixture = LabMonkey.labMonkeyBuilder()
.objectIntrospector(ConstructorPropertiesArbitraryIntrospector.INSTANCE)
.plugin(new JavaxValidationPlugin())
.build();

Through the fluent API plugin(), you can easily add plugins. By adding JavaxValidationPlugin, you can apply Java Bean Validation functionality to create objects.

It seems like a kind of decorator pattern, making it easier to develop and apply various third-party plugins.

public record LottoRecord(
@Min(1)
@Max(45)
int number
) {
public LottoRecord {
if (number < 1 || number > 45) {
throw new IllegalArgumentException("The lotto number must be between 1 and 45.");
}
}
}
@RepeatedTest(100)
void shouldBetween1to45() {
LottoRecord lottoRecord = fixture.giveMeOne(LottoRecord.class);

assertThat(lottoRecord.number()).isBetween(1, 45);
}

Conclusion

Most of the areas that were mentioned as lacking in the previous post have been improved, and I am very satisfied with using it. But somehow, the documentation2 seems a bit lacking compared to before...

Reference

Footnotes

  1. FixtureMonkey 0.3.0 - Object Creation Strategy

  2. FixtureMonkey

Using Date Type as URL Parameter in WebFlux

· 5 min read
Haril Song
Owner, Software Engineer at 42dot

Overview

When using time formats like LocalDateTime as URL parameters, if they do not match the default format, you may encounter an error message like the following:

Exception: Failed to convert value of type 'java.lang.String' to required type 'java.time.LocalDateTime';

What settings do you need to make to allow conversion for specific formats? This article explores the conversion methods.

Contents

Let's create a simple sample example.

public record Event(
String name,
LocalDateTime time
) {
}

This is a simple object that contains the name and occurrence time of an event, created using record.

@RestController
public class EventController {

@GetMapping("/event")
public Mono<Event> helloEvent(Event event) {
return Mono.just(event);
}

}

The handler is created using the traditional Controller model.

tip

In Spring WebFlux, you can manage requests using Router functions, but this article focuses on using @RestController as it is not about WebFlux.

Let's write a test code.

@WebFluxTest
class EventControllerTest {

@Autowired
private WebTestClient webTestClient;

@Test
void helloEvent() {
webTestClient.get().uri("/event?name=Spring&time=2021-08-01T12:00:00")
.exchange()
.expectStatus().isOk()
.expectBody()
.jsonPath("$.name").isEqualTo("Spring")
.jsonPath("$.time").isEqualTo("2021-08-01T12:00:00");
}

}

image1

When running the test code, it simulates the following request.

$ http localhost:8080/event Accept=application/stream+json name==Spring time==2021-08-01T12:00
HTTP/1.1 200 OK
Content-Length: 44
Content-Type: application/stream+json

{
"name": "Spring",
"time": "2021-08-01T12:00:00"
}

If the request is made in the default format, a successful response is received. But what if the request format is changed?

image2

image3

$ http localhost:8080/event Accept=application/stream+json name==Spring time==2021-08-01T12:00:00Z
HTTP/1.1 500 Internal Server Error
Content-Length: 131
Content-Type: application/stream+json

{
"error": "Internal Server Error",
"path": "/event",
"requestId": "ecc1792e-3",
"status": 500,
"timestamp": "2022-11-28T10:04:52.784+00:00"
}

As seen above, additional settings are required to receive responses in specific formats.

1. @DateTimeFormat

The simplest solution is to add an annotation to the field you want to convert. By defining the format you want to convert to, you can request in the desired format.

public record Event(
String name,

@DateTimeFormat(pattern = "yyyy-MM-dd'T'HH:mm:ss'Z'")
LocalDateTime time
) {
}

Running the test again will confirm that it passes successfully.

info

Changing the request format does not change the response format. Response format changes can be set using annotations like @JsonFormat, but this is not covered in this article.

While this is a simple solution, it may not always be the best. If there are many fields that need conversion, manually adding annotations can be quite cumbersome and may lead to bugs if an annotation is accidentally omitted. Using test libraries like ArchUnit1 to check for this is possible, but it increases the effort required to understand the code.

2. WebFluxConfigurer

By implementing WebFluxConfigurer and registering a formatter, you can avoid the need to add annotations to each LocalDateTime field individually.

Remove the @DateTimeFormat from Event and configure the settings as follows.

@Configuration
public class WebFluxConfig implements WebFluxConfigurer {

@Override
public void addFormatters(FormatterRegistry registry) {
DateTimeFormatterRegistrar registrar = new DateTimeFormatterRegistrar();
registrar.setUseIsoFormat(true);
registrar.registerFormatters(registry);
}
}
danger

Using @EnableWebFlux can override the mapper, causing the application to not behave as intended.2

Running the test again will show that it passes without any annotations.

image4

Applying Different Formats to Specific Fields

This is simple. Since the method of directly adding @DateTimeFormat to the field takes precedence, you can add @DateTimeFormat to the desired field.

public record Event(
String name,

LocalDateTime time,

@DateTimeFormat(pattern = "yyyy-MM-dd'T'HH")
LocalDateTime anotherTime
) {
}
    @Test
void helloEvent() {
webTestClient.get().uri("/event?name=Spring&time=2021-08-01T12:00:00Z&anotherTime=2021-08-01T12")
.exchange()
.expectStatus().isOk()
.expectBody()
.jsonPath("$.name").isEqualTo("Spring")
.jsonPath("$.time").isEqualTo("2021-08-01T12:00:00")
.jsonPath("$.anotherTime").isEqualTo("2021-08-01T12:00:00");
}

image5

tip

When the URI becomes long, using UriComponentsBuilder is a good approach.

String uri = UriComponentsBuilder.fromUriString("/event")
.queryParam("name", "Spring")
.queryParam("time", "2021-08-01T12:00:00Z")
.queryParam("anotherTime", "2021-08-01T12")
.build()
.toUriString();

Conclusion

Using WebFluxConfigurer allows for globally consistent formats. If there are multiple fields across different classes that require specific formats, using WebFluxConfigurer is much easier than applying @DateTimeFormat to each field individually. Choose the appropriate method based on the situation.

  • @DateTimeFormat: Simple to apply. Has higher precedence than global settings, allowing for targeting specific fields to use different formats.
  • WebFluxConfigurer: Relatively complex to apply, but advantageous in larger projects where consistent settings are needed. Helps prevent human errors like forgetting to add annotations to some fields compared to @DateTimeFormat.
info

You can find all the example code on GitHub.

Reference

Footnotes

  1. ArchUnit

  2. LocalDateTime is representing in array format

Precautions when using ZonedDateTime - Object.equals vs Assertions.isEqualTo

· 3 min read
Haril Song
Owner, Software Engineer at 42dot

Overview

In Java, there are several objects that can represent time. In this article, we will discuss how time comparison is done with ZonedDateTime, which is one of the objects that contains the most information.

Different but the same time?

Let's write a simple test code to find any peculiarities.

ZonedDateTime seoulZonedTime = ZonedDateTime.parse("2021-10-10T10:00:00+09:00[Asia/Seoul]");
ZonedDateTime utcTime = ZonedDateTime.parse("2021-10-10T01:00:00Z[UTC]");

assertThat(seoulZonedTime.equals(utcTime)).isFalse();
assertThat(seoulZonedTime).isEqualTo(utcTime);

This code passes the test. Although equals returns false, isEqualTo passes. Why is that?

In reality, the two ZonedDateTime objects in the above code represent the same time. However, since ZonedDateTime internally contains LocalDateTime, ZoneOffset, and ZoneId, when compared using equals, it checks if the objects have the same values rather than an absolute time.

Therefore, equals returns false.

image1 ZonedDateTime#equals

However, it seems that isEqualTo works differently in terms of how it operates in time objects.

In fact, when comparing ZonedDateTime, isEqualTo calls ChronoZonedDateTimeByInstantComparator#compare instead of invoking ZonedDateTime's equals.

image2

image3 Comparator#compare is called.

By looking at the internal implementation, it can be seen that the comparison is done by converting to seconds using toEpochSecond(). This means that it compares absolute time through compare rather than comparing objects through equals.

Based on this, the comparison of ZonedDateTime can be summarized as follows:

equals : Compares objects

isEqualTo : Compares absolute time

Therefore, when comparing objects that include ZonedDateTime indirectly, equals is called, so if you want to compare based on the absolute value of ZonedDateTime, you need to override the equals method inside the object.

public record Event(
String name,
ZonedDateTime eventDateTime
) {
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
Event event = (Event) o;
return Objects.equals(name, event.name)
&& Objects.equals(eventDateTime.toEpochSecond(), event.eventDateTime.toEpochSecond());
}

@Override
public int hashCode() {
return Objects.hash(name, eventDateTime.toEpochSecond());
}
}
@Test
void equals() {
ZonedDateTime time1 = ZonedDateTime.parse("2021-10-10T10:00:00+09:00[Asia/Seoul]");
ZonedDateTime time2 = ZonedDateTime.parse("2021-10-10T01:00:00Z[UTC]");

Event event1 = new Event("event", time1);
Event event2 = new Event("event", time2);

assertThat(event1).isEqualTo(event2); // pass
}

Conclusion

  • If you want to compare absolute time when equals is called between ZonedDateTime, you need to convert it, such as using toEpochSecond().
  • When directly comparing ZonedDateTime with isEqualTo in test code or similar scenarios, equals is not called, and internal conversion is performed, so no separate conversion is needed.
  • If there is a ZonedDateTime inside an object, you may need to override the object's equals method as needed.

Operating Jenkins with Docker

· 3 min read
Haril Song
Owner, Software Engineer at 42dot

Overview

This article explains how to install and operate Jenkins using Docker.

Contents

Install

Docker

docker run --name jenkins-docker -d -p 8080:8080 -p 50000:50000 -v /home/jenkins:/var/jenkins_home -u root jenkins/jenkins:lts 

Mount a volume to persist Jenkins data on the host machine. Unlike TeamCity, Jenkins manages all configurations in files. Setting up a mount makes authentication information and data management much more convenient, so be sure to configure it. Common target paths are /home/jenkins or /var/lib/jenkins.

For the purpose of this article, it is assumed that the path /home/jenkins has been created.

Authentication

To ensure security and access control for both the master and nodes, create a user named 'jenkins' and proceed as follows.

Setting User Access Permissions

chown -R jenkins /var/lib/jenkins

Managing SSH Keys

If you don't have keys, generate one using ssh-keygen to prepare a private key and a public key.

When prompted for a path, enter /home/jenkins/.ssh/id_rsa to ensure the key is created under /home/jenkins/.ssh.

GitLab

In GitLab's personal settings, there is an SSH setting tab. Add the public key.

When selecting Git in the pipeline, a repository path input field is displayed. Entering an SSH path starting with git@~ will show a red error. To resolve this, create a credential. Choose SSH credential to create one, and the ID value can be a useful value, so it is recommended to enter it.

Node Configuration

Nodes are a way to efficiently distribute Jenkins roles.

To communicate with the node, generate a key on the master using ssh-keygen. If you already have one that you are using, you can reuse it.

image

  • ID: This value allows Jenkins to identify the SSH key internally, making it easier to use credentials in Jenkinsfiles, so it's best to set a meaningful value. If not set, a UUID value will be generated.
  • Username: The Linux user. Typically, 'jenkins' is used as the user, so enter 'jenkins'. Be cautious as not entering this may result in a reject key error.

Docker Access Permissions

If the docker group does not exist, create it. Usually, it is automatically created when installing Docker.

sudo groupadd docker

Grant Jenkins user permission to run Docker by running the following command.

sudo gpasswd -a jenkins docker
# Adding user jenkins to group docker
sudo chmod 666 /var/run/docker.sock

Restart the Docker daemon to apply the changes.

systemctl restart docker

You should now be able to run the docker ps command.

Restart

When updating Jenkins version or installing, removing, or updating plugins, Jenkins restarts. However, if you are managing it with Docker, the container goes down, preventing Jenkins from starting. To enable restart, you need to set a restart policy on the container.

docker update --restart=always jenkins-docker

After this, the jenkins-docker container will always remain in a running state.

Caution

When updating plugins, carefully check if they are compatible with the current version of Jenkins in operation. Mismatched versions between Jenkins and plugins can often lead to pipeline failures.

Reference

Managing Jenkins with Docker

Making 'diff' More Intuitive, Difftastic

· One min read
Haril Song
Owner, Software Engineer at 42dot

Overview

Difftastic is a tool designed to make using git diff more convenient. It can be very useful for those who frequently use the git diff command in the terminal.

Usage

brew install difftastic

Global setting:

git config --global diff.external difft

Now, when you use the git diff command, you can see much more intuitive diff results compared to before.

image

Reference

Could not find a valid Docker environment

· 2 min read
Haril Song
Owner, Software Engineer at 42dot

Overview

After updating my Mac and finding that Docker was not working properly, I had to reinstall it. However, I encountered an error where the container was not running properly when running tests.

It turned out that there was an issue with the /var/run/docker.sock not being properly configured. Here, I will share the solution to resolve this issue.

Description

This problem occurs in Docker desktop version 4.13.0.

By default Docker will not create the /var/run/docker.sock symlink on the host and use the docker-desktop CLI context instead. (see: https://docs.docker.com/desktop/release-notes/)

You can check the current Docker context using docker context ls, which will display something like this:

NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT                                KUBERNETES ENDPOINT                                 ORCHESTRATOR
default moby Current DOCKER_HOST based configuration unix:///var/run/docker.sock https://kubernetes.docker.internal:6443 (default) swarm
desktop-linux * moby unix:///Users/<USER>/.docker/run/docker.sock

To fix the issue, either set the default context or connect to unix:///Users/<USER>/.docker/run/docker.sock.

Solution

Try running the following command to switch to the default context and check if Docker works properly:

docker context use default

If the issue persists, you can manually create a symbolic link to resolve it with the following command:

sudo ln -svf /Users/<USER>/.docker/run/docker.sock /var/run/docker.sock

Reference

Key Generation Error

· One min read
Haril Song
Owner, Software Engineer at 42dot
info

Here is a simple solution to resolve the error.

key generation error: Unknown signature subpacket: 34

While trying to register a GPG key on Keybase, the above error occurred. In search of a solution, I found the following workaround on GitHub.

$ gpg --edit-key mykey

gpg> showpref
[ultimate] (1). mykey
Cipher: AES256, AES192, AES, 3DES
AEAD: OCB, EAX
Digest: SHA512, SHA384, SHA256, SHA224, SHA1
Compression: ZLIB, BZIP2, ZIP, Uncompressed
Features: MDC, AEAD, Keyserver no-modify

gpg> setpref AES256 AES192 AES 3DES SHA512 SHA384 SHA256 SHA224 SHA1 ZLIB BZIP2 ZIP
Set preference list to:
Cipher: AES256, AES192, AES, 3DES
AEAD:
Digest: SHA512, SHA384, SHA256, SHA224, SHA1
Compression: ZLIB, BZIP2, ZIP, Uncompressed
Features: MDC, Keyserver no-modify
Really update the preferences? (y/N) y

gpg> save

After this, the operation should run smoothly. For more details, refer to the provided link.

Reference

How to Change Vimium Shortcuts

· One min read
Haril Song
Owner, Software Engineer at 42dot

Overview

Recently, as I started using Vim, I've been aligning all my environments with Vim. Among them, I noticed that there are some differences in the shortcuts between Vimari, the Vim extension for Safari, and Vimium, the extension for Chrome. To unify them, I decided to remap specific keys. In this guide, I will introduce how to remap shortcuts in Vimium.

Vimium Options Window

where

Click the button in the Chrome extension to open the options.

input

By modifying this section, you can change the shortcuts. The basic mapping method is the same as in Vim. Personally, I found it more convenient to change the tab navigation shortcuts from q, w in Vimari to J, K in Vimium.

If you're unsure which key to map to a specific action, you can click on "show available commands" next to it for a helpful explanation.

help-view

From here, you can find the desired action and map it to a specific key.

Enable Keyboard Key Repeat on Mac

· One min read
Haril Song
Owner, Software Engineer at 42dot

On a Mac, when you press and hold a specific key for a while, a special character input window like umlauts may appear. This can be quite disruptive when using editors like Vim for code navigation.

defaults write -g ApplePressAndHoldEnabled -bool false

After running the above command and restarting the application, the special character input window, such as umlauts, will no longer appear and key repetition will be enabled.