Pressure makes diamonds, as the saying goes. I worked on a high-pressure project for a couple of weeks (as in, it needed to be done before we even started), and these are some of the lessons we learned as a team. The lessons are mostly tips and tricks, as we learned a lot on the job.
General lessons learned
Way of working
Bring (at least) two developers to the project. One will focus on the algorithm, the other will focus on the code quality and support as much as possible. Notice the choice of words: “focus”. This means that all developers do all the things, but their main task is different.
Don’t underestimate the impact of code quality. Code should be as clear as possible, so that it doesn’t get in the way of solving the business problem. When you’re constantly thinking about what the code does, you’re not thinking about how to solve the business problem. On that note, the first versions were set up as procedural. Refactor to object oriented. OO has advantages over procedural, and it would be a waste to not have access to those advantages. This refactoring was well worth the effort, as we had our codebase audited. No major flaws were encountered during the audit.
Version control
Get a version control tool in place, and choose the one that is easiest to use. You can share code by emailing .zip files, but that’s too cumbersome. Besides, errors get made. Use git, ask around how to do that, and ignore project managers who tell you not to do this. Even a paid github repository is better than nothing.
maven
Manually include dependencies
It is possible to add dependencies to the build, without the need for those dependencies to be available in a repository. You’ll include them from a /lib folder or something like that:
<dependency>
<groupId>group.id</groupId>
<artifactId>artifact</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>${project.basedir}/src/test/resources/to-include.jar</systemPath>
</dependency>
Create complete jar
To build the resulting jar with dependencies, use the following command:
mvn assembly:assembly -DdescriptorId=jar-with-dependencies
Version tracking
Resource filtering, to update variables in your resources with maven properties. But only variables in certain files, all other files should not be filtered because that might corrupt them:
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>false</filtering>
</resource>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>project-version.properties</include>
</includes>
</resource>
</resources>
</build>
Contents of project-version.properties:
version = ${build.version}
where ${build.version} is a property in the pom file, along with the format for this timestamp:
<properties>
<maven.build.timestamp.format>yyyyMMdd-HHmm</maven.build.timestamp.format>
<build.version>${maven.build.timestamp}</build.version>
</properties>
Download sources
To download all sources from the dependencies (when available), type
mvn dependency:sources
This will allow you to inspect the actual source code when you’re in a debugging session.
Skip tests
There are two ways of skipping unit tests:
mvn -DskipTests <task>
Only skips _executing_ the tests. The unit tests will still be compiled
mvn -Dmaven.test.skip=true
Does not compile the tests, and therefore the tests are not executed.
One piece of software
For testing purposes, we made our program so it ran locally. The same program could run, without modifications, on the server. We used hard-coded paths and keys for the server version, with fallbacks for the local standalone version. This allowed us to focus on the algorithms, and find/fix environments issues quite fast.
Patching jars
We had to patch the Mendelson jars a few times, before we decided to create a maven build script for the source code.
javac -classpath <jar to be patched>;<jars containing non-local classes used by the class to be compiled> path\to\modified\file.java
Then open the jar with a zip-tool (7zip, for example), and replace the old class with the newly compiled version.
Logging
Add as much logging as useful. This is probably more than you think. In our case, logging wasn’t showing up. So we wrote a LoggingFacade which wrote its output to the default logging framework, AND to System.out or System.err if needed.
Debugging
Debugging will provide more information than logging, but is not always possible.
Make one version that run standalone, so you can attach a debugger while developing.
Make sure you can remotely debug the server. Start the server with debug enabled, with the following command-line parameter:
-agentlib:jdwp=transport=dt_socket,address=localhost:4000,server=y,suspend=y
This starts the program in debug mode, listening to debuggers on TCP port 4000. You can choose any port that is convenient for you.
You might need to open an SSH tunnel to your server, listening locally to port 4000, and forwarding it to localhost:4000. Notice that localhost is the localhost of the server, not the localhost from which you make the connection to the server.
Then configure your IDE to connect to a remote application.
Spring-Boot
One of the avenues we’ve explored was to build a standalone program to intercept and process the messages in a more controllable way. Spring-Boot was introduced for this, but not continued. It is worth exploring these kinds of avenues when you’re stuck, because they might give some insight in how to continue.
Spring-Boot offers quite a lot of extras that we can use for our project, such as a standalone server (run with mvn spring-boot:run). Any application can still be run from within the IDE, because the applications still have a main() function.
Links:
About the producing service: https://spring.io/guides/gs/producing-web-service/
About the consuming service: https://spring.io/guides/gs/consuming-web-service/
Switching from application: https://stackoverflow.com/questions/23217002/how-do-i-tell-spring-boot-which-main-class-to-use-for-the-executable-jar
To test the producing service, use Postman (https://www.getpostman.com/apps)
The service can be reached with a POST request on http://localhost:8080
Headers: content-type: text/xml
Body type: raw
Body contents can be found on the producing link, file is called “request.xml”
Project specific
Decrypting XML
The XML might have been encrypted with a cipher that isn’t available to you. Find the correct cipher in the following section:
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2009/xmlenc11#rsa-oaep"> <ds:DigestMethod xmlns:ds="http://www.w3.org/2000/09/xmldsig#" Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/> <xenc11:MGF xmlns:xenc11="http://www.w3.org/2009/xmlenc11#" Algorithm="http://www.w3.org/2009/xmlenc11#mgf1sha256"/> </xenc:EncryptionMethod>
Take special note of the Digest Method and the Mask Generation Function, as these might not be available to you. You need to use a third party library that implements the exact cipher that is used. In our case that is Apache Santuario.
Initializing Santuario
Santuario must be initialized before it’s used. However, before initializing the main cryptography engine, the Internationalization framework needs to be initialized. Normally this is initialized with the locale en-US, but only the en (without the _US_ part) properties file is available. This should not be a problem, since this properties file is part of a fallback mechanism. However, in our case, this fallback mechanism doesn’t work.
First initialize Santuario with an empty resource bundle, then initialize the cryptography engine.
Binary data
In one instance of our project, the binary file had a repeating sequence EF BF BD. This is caused by creating a String from the binary data, and requesting the bytes from that String. Strings and binary aren’t the best of friends, keep them separated!
