Lint your shell scripts

I will start this by quoting the bashstyle's readme:

Bash is like the JavaScript of systems programming. Although in some cases it's better to use a systems language like C or Go, Bash is actually an ideal systems language for many smaller POSIX-oriented or command line tasks. Here's three quick reasons why:

  • It's everywhere. Like JavaScript for the web, Bash is already there ready for systems programming.
  • It's neutral. Unlike Ruby, Python, JavaScript, or PHP, Bash offends equally across all communities. ;)
  • It's made to be glue. Write complex parts in C or Go (or whatever!), and glue them together with Bash.

My concern about that is the quality of the code itself. Like JavaScript, most people will just google and do what they want in the first way that they found - quick and dirty. You can blame me too.

So, projects like bashstyle are important to set a common sense in how to do things using bash. The problem is that it's not automated.

A project that really helps with that is shellcheck. It's an executable written in Haskell, which can lint your scripts (in bash, zsh, and others). Sure enough, we can put this in a Countinuous Integration system and watch it do the validation for us.

An simple example of build.sh script is:

#!/bin/bash
set -eo pipefail
[[ "${DEBUG:-}" ]] && set -x

main() {
  find . -type f -perm +111 | grep -v "\.git" | while read script; do
    shellcheck "$script"
  done
}

main "$@"

Pretty straightforward to integrate with travis-ci too (.travis.yml):

language: bash
install:
  - wget http://ftp.debian.org/debian/pool/main/s/shellcheck/shellcheck_0.3.4-3_amd64.deb
  - sudo dpkg -i shellcheck_0.3.4-3_amd64.deb
script:
  - ./build.sh

I'm already using this in my dotfiles, and found some really stupid mistakes to fix. Sure thing, this is an awesome tool!

Wanna see it in the so said real world? Check my build.sh here and my .travis.yml here.

Hope you like it!

Comment this post here.

Don't use junit.framework

Every once in a while I see somewhere a wrong import to the old, deprecated junit.framework instead of org.junit.

I particularly believe they should remove this package and put it in a junit-compat jar or something... but, while they don't do that, we can avoid those old imports with 2 basic steps:

1. Replace all old imports with the new imports

This is pretty straightforward with a simple bash script:

#!/bin/bash
for file in $(git grep --break --heading "import junit." | grep java); do
  sed -i.bak s/junit.framework/org.junit/g $file
  rm -rf $file.bak
done

2. Don't accept it in new code

The second step is basically to break the build whenever someone try to use those old imports. A simple way to do that is using the restrict-maven-plugin.

An example of configuration for maven will look like this:

<plugin>
  <groupId>com.yamanyar</groupId>
  <artifactId>restrict-maven-plugin</artifactId>
  <version>0.6</version>
  <executions>
    <execution>
      <phase>process-classes</phase>
      <goals>
        <goal>restrict</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <continueOnError>false</continueOnError>
    <restrictions>
      <restriction>com.carlosbecker.* to junit.*</restriction>
    </restrictions>
  </configuration>
</plugin>

Sure it is a simple issue, which might never cause you problems, but, if one day the Junit team remove the old packages (probably soon), you will have a headache. Besides that, writing new code using deprecated classes?

Comment this post here.

Notify your team using Telegram

In an ideal world, applications will never go down, for example. In the real world, shit happens. Every second counts.

Let's take the example of a server going down for some reason (which shouldn't happen, ever). I want the team to be notified as soon as possible to mitigate the issue. To do that in a easy and free way, I decided to use Telegram.

Wait, wait... what is this Telegram thing again?

According to their website:

Telegram is a cloud-based mobile and desktop messaging app with a focus on security and speed.

It's basicaly a Whatsapp-like messenger, with an open API and more security.

Wiring it up

First, install the telegram-client following their README.

Then, write your script. Mine looks like this:

#!/bin/bash
TG=/opt/tg

# $1: Server Name (used in the message body)
# $2: URL to test
# $3: Group/User to notify
function check() {
    wget -q $2 -O /dev/null
    if [ ! $? -eq 0 ]; then
        echo "msg $3 WARN: $1 IS DOWN!!!" | $TG/bin/telegram-cli \
            -k $TG/tg-server.pub -W
    fi
}

check "my server" "http://myserver.blah.fake.address/check" "Server1"
check "another server" "http://myserver2.blah.fake.address/check" "Server2"
check "Google" "http://google.com" "General"

Sure, you can script it in order to notify you about anything, including some business specific things, dependency on third-party systems... well, use your imagination.

I also added it to the crontab, so it will run every minute.

And, sure enough, it works:

telegram screenshot

Hope it helps!

Comment this post here.

Using JUnit Rules to simplify your tests

Have you ever wrote JUnit tests extending a class that does some before and after work, so you didn't have to repeat that code in various test classes? Well, I will not say that you have been doing it wrong, but, sure enough, you could do it better. How? Using JUnit Rules!

The Basics

Well, before we learn all that, let's start with the basics, shall we?

Timeouts

Let's take a simple example: Suppose that you want to set a timeout for all test methods in a given class, an easy way to do that is like this:

public class BlahTest {
    @Test(timeout = 1000)
    public void testA() throws Exception {
        // ...
    }

    @Test(timeout = 1000)
    public void testB() throws Exception {
        // ...
    }

    @Test(timeout = 1000)
    public void testC() throws Exception {
        // ...
    }

    @Test(timeout = 1000)
    public void testD() throws Exception {
        // ...
    }

    @Test(timeout = 1000)
    public void testE() throws Exception {
        // ...
    }

    // ...
}

Besides that you repeated yourself tons of times, if you want to change this timeout, you will have to change it in all methods. There is no need to do that. Just use the Timeout Rule:

public class BlahTest {
    @Rule
    public Timeout timeout = new Timeout(2000);

    @Test
    public void testA() throws Exception {
        // ...
    }

    @Test
    public void testB() throws Exception {
        // ...
    }

    @Test
    public void testC() throws Exception {
        // ...
    }

    @Test
    public void testD() throws Exception {
        // ...
    }

    @Test
    public void testE() throws Exception {
        // ...
    }

    // ...
}

Temporary Folder

Have you ever needed to do some test that uses File and/or needed a temporary file/folder? TemporaryFolder to the rescue:

public class BlahTest {
    @Rule
    public TemporaryFolder tempFolder = new TemporaryFolder();

    @Test
    public void testIcon() throws Exception {
        File icon = tempFolder.newFile("icon.png");
        // do something else...
    }
}

Expected Exceptions

Ever needed more control on exceptions? Try the ExpectedException rule:

public class BlahTest {
    @Rule
    public ExpectedException exception = ExpectedException.none();

    @Test
    public void testIcon() throws Exception {
        exception.expect(IllegalArgumentException.class);
        exception.expectMessage("Dude, this is invalid!");
        // do something that you expect to throw an exception...
    }
}

Custom Rules

That's neat, but... what if you need something else... something more "custom"? Well, you can implement your own rules by implementing the TestRule interface, for example, a Rule that init Mockito mocks (not very useful):

@RequiredArgsConstructor
public class MockRule implements TestRule {
  private final Object target;

  public Statement apply(Statement base, Description description) {
    return new Statement() {
      @Override
      public void evaluate() throws Throwable {
        MockitoAnnotations.initMocks(target);
        base.evaluate();
      }
    };
  }
}

To use it, you just need to declare that rule in your test class:

public class BlahTest {
    @Rule
    public MockRule mock = new MockRule(this);

    @Mock
    private BlahService service;

    @Test
    public void testBlah() throws Exception {
        Assert.assertThat(
            service.blah(),
            CoreMatchers.notNullValue()
        );
    }
}

External Resources

Returning to the example of this post's first paragraph, you can also have custom external resources rules by extending the ExternalResource class:

public class MyServer extends ExternalResource {
  @Override
  protected void before() throws Throwable {
    // start the server
  }

  @Override
  protected void after() {
    // stop the server
    }
}

I believe that this makes more sense with Integration Tests, though. Also, in this case, you probably would not want/need to start and stop the server before and after each test method, righ? So, you can use the @ClassRule annotation:

public class BlahServerTest {
    @ClassRule
    public static MyServer server = new MyServer();

    @Test
    public void testBlah() throws Exception {
        // test something that depends on the server.
    }
}

Attention: Note that when you use @ClassRule, your rule instance should be static, just like @BeforeClass and @AfterClass methods.

Going Further

That's the basics that will save you tons of abstract classes and ugly code. I would also recommend you to take a good read at the junit wiki. If you have any question, don't exitate to comment bellow, I will surely try to answer them. :beer:

Comment this post here.

Exponential Backoff with Java 8

Exponential backoff is an algorithm that uses feedback to multiplicatively decrease the rate of some process, in order to gradually find an acceptable rate. - Wikipedia

I recently used this strategy in work to deal with another service that we need to integrate. Sometimes, the service will just refuse the connection, without any reason. If I keep pushing, it will, someday, accept it.

So, I used Java 8 Functional Interfaces to implement this in a not-so-ugly way, also using a Fibonacci's Sequence to incremente the wait time:

The ExponentialBackOffFunction Functional Interface:

import java.rmi.RemoteException;

@FunctionalInterface
public interface ExponentialBackOffFunction<T> {
    T execute();
}

The ExponentialBackOff main class:

import static java.util.Arrays.asList;

import java.net.SocketTimeoutException;
import java.util.List;

import javax.net.ssl.SSLHandshakeException;

import lombok.extern.log4j.Log4j;

@Log4j
public final class ExponentialBackOff {
    private static final int[] FIBONACCI = new int[] { 1, 1, 2, 3, 5, 8, 13 };
    private static final List<Class<? extends Exception>> EXPECTED_COMMUNICATION_ERRORS = asList(
            SSLHandshakeException.class, SocketTimeoutException.class );

    private ExponentialBackOff() {

    }

    public static <T> T execute(ExponentialBackOffFunction<T> fn) {
    for (int attempt = 0; attempt < FIBONACCI.length; attempt++) {
            try {
                return fn.execute();
            } catch (Exception e) {
                handleFailure( attempt, e );
            }
        }
        throw new RuntimeException( "Failed to communicate." );
    }

    private static void handleFailure(int attempt, RemoteException e) {
        if (e.getCause() != null && !EXPECTED_COMMUNICATION_ERRORS.contains( e.getCause().getClass() ))
            throw new RuntimeException( e );
        doWait( attempt );
    }

    private static void doWait(int attempt) {
        try {
            Thread.sleep( FIBONACCI[attempt] * 1000 );
        } catch (InterruptedException e) {
            throw new RuntimeException( e );
        }
    }
}

Usage:

ExponentialBackOff.execute( () -> work() );

This will try to execute the work method incrementing the time between each call that fail with an expected error.

Comment this post here.