Find the slowest tests of a Java project

I found that it's pretty hard to have a project with high test coverage and fast build... if the tests are slow, people will feel the need to skip them to speed up the build, and will probably write less tests than they should, afraid that the build will became even slower. You go out for a walk and when you come back no one is running or writing tests anymore...

One way to avoid that is to track down the slowest tests and fix them. The less dependencies your test have, the fast it will run. Just saying.

Paraphrasing Uncle Bob:

"The first rule about tests is that they should be fast. The second rule about tests is that they should be faster than that."

** Actually he said that about method sizing, but, I think this fits just fine.

To help with that, I wrote a small script to generate a list of the slowest JUnit tests. It should give you some direction in where to attack. The script is pretty simple, but it does the job.

git clone https://gist.github.com/9420690.git scripts
scripts/slowest-tests project/folder

You can also generate that list in CSV format by running:

scripts/csv-slowest-tests project/folder

More than that, you can use maven-profiler to find other slow parts of your build and fix them.

One last tip: if are having problems with low test coverage in your project, try coverage-maven-plugin, a maven plugin that will blame pull request with coverage bellow a specified amount.

That's it for today, happy hacking!

Comment this post here.

Spotify for Everyone

I personally love Spotify. I can hear music everywhere and even sync some music to my devices, and it works pretty well even in slow connections.

But in lot of countries, including Brazil (where I currently live), Spotify would not let you create an account, due to some copyright bullshit.

Lucky for us, this is one case where proxies are actually useful. To begin, you will need a proxy server in USA, or, you could just open a SSH connection to some server in US and use it as a SOCKS5 proxy.

I used the later and it is working as expected for a month or two now. So, I will guide you through this process.

1. Get some server

First thing, you need a server in US. I recommend DigitalOceal for that. It is really cheap, easy to signup, easy and fast to create and destroy the server.

Create an account and create a server. You can put anything you want in the hostname and leave the rest as it is.

You will receive an email with the password to access your server, and it will be up and running in about 60 seconds.

2. Create a SOCK5 proxy

You will receive from DigitalOcean an email like this:

You can access it using the following credentials:
IP Address: X.X.X.X
Username: root
Password: pwd

To create the proxy, you will need OpenSSH installed. It is default in Linux and Mac OS X boxes, I don't know about Windows, but, given the history, probably not. In that case, you are by yourself to get this working.

Now, back to what really matters: to create the proxy server, run this:

ssh root@X.X.X.X -NCD 8080

Here, X.X.X.X is the IP from the DigitalOcean email. When asked, enter also the password provided in the very same email.

Now you should have a proxy server running!

3. Setup your machine

You will have to setup SOCKS proxy to address to 127.0.0.1 and the port to 8080.

I recommend you to use Mozilla Firefox for that, so you can change the settings for the browser only.

How to setup the proxy in Firefox

Don't forget to check the SOCKSv5 option if asked and to uncheck the "Use this proxy server for all protocols" if it is checked.

Now, point to google and search for the term "my ip". You should see the IP you received by email.

4. Create the account, finally

Now, just point the browser to the Spotify website and register. You will notice that you will not be able to change your country. This is expected. Just proceed and create your account.

Now:

  • Go to the console where you ran that command I said before, and press CTRL+C. This will stop the proxy server;
  • Go to Firefox settings and check the "No Proxy" option.

Now, access your Spotify Payment Method Setup and configure everything. Yes, you will have to pay or they will block you out after some time.

5. Enjoy!

Download the app for your OS or access the web player and enjoy your music!

6. Clean up

If you will not need the server anymore, you can shut it down. To do that, access your droplet in DigitalOcean, click in the destroy tab and follow the instructions.

Thats all folks

Hope you enjoy this not-very-technical post. Cheers!

Comment this post here.

PullRequest Coverage Blammer Maven Plugin

At the company I work Pull Requests are part of our culture. When someone opens a Pull Request, we do Code Review. If we think it's ok, we comment "+1", or "-1" otherwise. We usually only merge a PR when it has 3 or more "+1" comments.

Part of this review is to check for tests. We used to manually look at our code coverage statuses, and see if our recently added lines are with enough coverage. But this is boring, and we are developers, and developers automate things, and so we did.

@velo came with the idea of doing a maven plugin to automatically report bad code coverage in new lines added in pull requests. In the weekend, I sit next to a pack of beers and a considerable amount of coffee and made it work. Well, not the 100% one perfect solution, but we are improving it. And it is OpenSource! You can see a Pull Request example here. All my comments in this PR are actually the plugin working via the Github API :heart:.

coverage

We already have Sonar and Code Formatter Maven Plugins (both written by @velo) that will fail the build in case of code style changes and report code with bugs and other issues, respectively, so, it makes sense to us to add coverage to the party.

We configured those plugins to run within our CI Server, so, everything is automated: we just open a pull request and the CI do the rest.

Now, with test coverage reports automated for new code, we only need to discuss about what really matters: the business.

Comment this post here.

2013

So, just like in 2012, I decided to write a little review of this year that end.

Unlikely 2012, 2013 was really calmer. Besides some side projects not yet revealed to the public (but I hope I can reveal soon), not much has changed.

  • I was promoted in my current Job;
  • I improved my skills in several languages;
  • Bought my first Mac (Macbook Pro mid 2012);
  • Was robbed: The macbook (which I had for 2 months), my old Dell Inspirion and my wife's HP laptop;
  • Bought a new Mac (Macbook Pro late 2012);
  • Bought a Kindle (probably one of my bests acquisitions ever);
  • Bought a new Acoustic Guitar;
  • As you can imagine, I now have a debt in bank, which I will destroy this year (the debt, not the bank -yet-);
  • Opensourced several projects;
  • Worked with some friends in other projects;
  • First Flight (to Sao Paulo, twice);
  • Participated in 24 pull requests for christmas, but wasn't able to do all the 24 pull requests;
  • Started learning a few languages, such as Scala and SML;
  • Attended to some events in Joinville - SC, one in Florian√≥polis - SC and one in Sao Paulo - SP;
  • Started training Muay Thai.

So, is basically this. As I already said: calmer. But I want this to change again in 2014. I even created a small list of things I want to accomplish in 2014 to help me stay focused, let's see if it will work.

Well, I wish for you all an awesome year and hope you accomplish everything you want!

Comment this post here.

This site is now powered by Turbolinks

I think that turbolinks is great: it mades it easy to add AJAX PushState to your Rails Applications. The only problem with that is that we can't use it any WEB application, because it's a Ruby Gem. So I did some ugly-but-easy hacks and add it to this very site. I will describe the steps above.

First things first

  • This site is opensource.
  • I'm using Jekyll alongside with less, bower, grunt and of course, node and npm.
  • You can take a look at these commits to see what I did.

With that said, let's do the thing.

Getting turbolinks

Well, turbolinks is a rubygem, but it has a lot of dependencies, and Jekyll don't have the Rails Asset Pipeline, so I can't figure out how to use this way.

So, to get the pure turbolinks file, I used grunt-curl and compiled it to javascript using grunt-contrib-coffee:

curl: {
  turbolinks: {
    src: 'https://raw.github.com/rails/turbolinks/master/lib/assets/javascripts/turbolinks.js.coffee',
    dest: '_assets/turbolinks.coffee'
  }
},
coffee: {
  compile: {
    files: {
      '_assets/turbolinks.coffee.js': '_assets/turbolinks.coffee'
    }
  }
},
concat: {
  dist: {
    src: [
      // other JS files
      '_assets/turbolinks.coffee.js',
      '_assets/up.js'
    ],
    dest: 'js/up.min.js'
  }
},
uglify: {
  build: {
    src: 'js/up.min.js',
    dest: 'js/up.min.js'
  }
}

As you can see, I get always the last file from trunk and compile to plain old javascript. After that, I also concat it with other js files and uglify the result (using grunt-contrib-concat and grunt-contrib-uglify, respectively).

Dealing with style="background-image: url(image.jpg);"

For some reason I'm now quite sure, turbolinks mess up with this kind of style declaration (which I use in the image headers). The solution I've found is kinda weird, but it works:

var reloadImages = function() {
  var styles, style, url, _i, _len, _el;
  styles = Array.prototype.slice.call(document.body.querySelectorAll('[style]'));
  for (_i = 0, _len = styles.length; _i < _len; _i++) {
    _el = styles[_i]
    style = _el.getAttribute('style');
    if (!(style.indexOf('url(') > -1)) {
      continue;
    }
    url = style.match(/url\((.*)\)/)[1];
    _el.style.backgroundImage = 'url(' + url + ')';
  }
}
$(document).on('page:change', function() {
  reloadImages();
});

Dealing with the twitter button

The twitter button adds a script to the head, which turbolinks doesn't replace, so, it gets buggy. To fix that I changed a little the default button markup:

<a  href="https://twitter.com/share" class="twitter-share-button"
data-lang="en" data-size="large"
data-url="{{ site.production_url }}{{ page.url }}"
data-text="{{ page.title }}">
  Tweet
</a>
<script>
!function(d,s,id){
  var js,fjs=d.getElementsByTagName(s)[0];
  if(!d.getElementById(id)){
    js=d.createElement(s);
    js.id=id;
    js.src="//platform.twitter.com/widgets.js";
    fjs.parentNode.insertBefore(js,fjs);
  }
}(document,"script","twitter-wjs");
</script>

And add something at the page change event:

var twttr;
$(document).on('page:change', function() {
  if (twttr) {
    twttr.widgets.load();
  }
});

It was the best solution I've found.

Dealing with Disqus load on bottom

I changed the Disqus scripts to only load the commends when the user reach the bottom of the page:

<div id="disqus_thread">
  Loading Comments...
</div>
<script type="text/javascript">
  var disqus_loaded = false;
  var disqus_shortname = 'caarlos0blog';

  function load_disqus () {
    disqus_loaded = true;
    var dsq = document.createElement('script');
    dsq.type = 'text/javascript';
    dsq.async = true;
    dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
    (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
  }

  window.onscroll = function(e) {
    var   currentScroll = (window.innerHeight + window.scrollY)
        , elScroll = document.getElementById('disqus_thread').offsetTop;
    if (currentScroll >= elScroll && disqus_loaded == false) {
      load_disqus()
    }
  };
</script>
<noscript>Please enable JavaScript to view the <a href="http://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>

But it causes me some trouble with Turbolinks. To fix that, I simply did:

$(document).on('page:fetch', function() {
  window.onscroll = void 0;
});

And it worked!

Adding nprogress

I also decided to add nprogress, a lib that provides the loading bar medium-style (and youtube-style). I had to add it to my bower.json, add the import in my less file (with a (less) prefix, so it imports it as a less file) and mix it up in my js. I also had to bind the events to nprogress, like this:

$(document).on('page:fetch', function() {
  NProgress.start();
});

$(document).on('page:change', function() {
  NProgress.done();
});

$(document).on('page:restore', function() {
  NProgress.remove();
});

// hides the spinner
NProgress.configure({ showSpinner: false });

And it was working as expected.

The Final Countdown

The result is what you're seeing right now. I found it neat and it loads really faster. Hope you like it!

Comment this post here.