The tests are in place, so it's possible to write the actual bytecode transformations. ASM's user guide is a good reference for learning Java bytecode.
2011-12-27
2011-12-23
2011-12-21
2011-12-18
Let's Code Jumi #107: Testing Bytecode Manipulation (Part 1)
I will create a Java agent which does bytecode manipulation to check that non-thread-safe classes (i.e. classes annotated with JSR-305's @NotThreadSafe) are used only from one thread. Unit tests for bytecode manipulation can be written using a custom class loader, which transforms only the bytecode of a selected class, and a bunch of "guinea pig" classes [1] to be used as test data.
[1] See GOOS page 284.
2011-12-09
Let's Code Jumi #103: Parameterized Tests (Part 1)
We need to create a new module for a Java agent, so now that we touch the build tests, it's a good idea to refactor them into parameterized tests. That should make them more maintainable. That will also allow me to write tests for the test data, to make sure that all of the artifacts follow some project principles.
2011-11-30
Let's Code Jumi #99: Using Code Generator (Part 1)
We can now take our code generator Maven plugin into use, which should weed out the remaining missing features. I also figured out how to configure Maven Invoker Plugin so that we can use JUnit and other libraries in the integration test scripts.
2011-11-14
Let's Code Jumi #93: Embedded Java Compiler (Part 1)
Our Maven plugin must use the Java compiler to compile the listener interface's source file, because the code generator must be able to use reflection on it. This means fiddling with class loaders and more integration with Maven to find out the project's classpath. Taking a look at the Maven Compiler Plugin is helpful. We will also see how to write post-build verification scripts for the Maven Invoker Plugin.
2011-11-01
Let's Code Jumi #88: Maven Plugin Testing (Part 1)
To get started on writing the Maven plugin, we first need some way to write tests for it. I found from Stack Overflow a list of options, of which I will evaluate the alternatives for integration testing: shitty-maven-plugin and maven-invoker-plugin. I choose the latter one and write an integration test using it.
2011-10-24
Let's Code Jumi #86: Generating the Events (Part 1)
Generating source code for the event classes whose instances will be passed through event queues.
2011-10-20
Let's Code Jumi #83: Generating the Backend (Part 1)
Generating source code for the backend class which will convert event instances into method calls.
2011-10-14
Let's Code Jumi #80: Generating the Frontend (Part 1)
Generating source code for the frontend class which will convert method calls into event instances.
2011-10-13
Let's Code Jumi #79: Generating the Factory
Generating source code for the factory class which will instantiate all the other generated classes.
2011-10-10
Let's Code Jumi #77: Code Generation (Part 1)
Creating classes which generate the Java source code of the event stubs based on an event listener interface. There will be one factory, one frontend and one backend class for the interface, plus one event class for each method of the interface.
2011-10-08
Let's Code Dimdwarf #69: Commit Requests
Deepening the design of our walking skeleton by introducing asynchronous messages for transaction commit requests.
2011-10-06
Let's Code Jumi #75: Hello Maven Plugin (Part 1)
After a quick test for the SuiteListener interface, the next big refactoring is to create a source code generator for the event stubs of the actors framework. It should be best implemented as a Maven plugin, which I have never done before, so it will be interesting to find out how to do it.
2011-10-05
Let's Code Jumi #74: New Domain
I registered the domain jumi.fi so I will need to change all packages and groupIds of the project. The build tests prove their worth once again by reminding about all the places which need to be updated.
2011-10-02
Let's Code Dimdwarf #68: Session API
Deepening the design of our walking skeleton by creating the application API for sending and receiving session messages.
2011-09-28
Let's Code Dimdwarf #67: Hide SessionHandle
The ClientSessions class is good enough for now, so we can resume doing what initially caused the need for it; we need to keep the SessionHandle private to the network layer and use only SessionId elsewhere.
2011-09-25
Let's Code Jumi #68: Actorified Runners (Part 8)
Finishing the work on the new actor-based runners, now that the actor framework has been improved to better match their needs.
2011-09-20
Let's Code Jumi #64: Refactoring Actor Tests (Part 1)
Putting the single-threaded actors into use and cleaning up the actor tests by extracting recurring abstractions. Also trying out an idea about extracting details into a super class which I read about recently.
2011-09-16
Let's Code Jumi #61: Single-Threaded Actors (Part 1)
Creating a second implementation of the Actors class which will be single-threaded and useful for unit testing. I will use abstract contract tests for the common behaviour which must work with both threading models. Also some tests need to be refactored to decoupled them from the threading model.
2011-09-13
Let's Code Dimdwarf #66: State Machine DSL
The code in ClientSessions has started to resemble a domain specific language for state machines with its operations, transitions and actions. Let's refactor it towards a DSL. After recording this episode I refactored it still some more off-camera. See these three last commits for the final DSL.
2011-09-08
Let's Code Jumi #58: Worker Threads (Part 1)
Implementing the actor-based runners got too complicated, but I have a hunch that it would be easier to implement them if I would have a simple way to start worker threads. They would run unattended after started and have a callback of when they finish. So let's improve the actor infrastructure.
2011-09-05
Let's Code Dimdwarf #65: Feature Complete
The state machine has now been implemented and most logic has been moved from the controller to the ClientSessions state machine, leaving the controller focused on just simple mapping. Only some refactoring remains.
2011-08-22
Let's Code Jumi #51: Actorified Runners (Part 1)
Converting the application to be actor-based. The TestRunCoordinator class which remains from the walking skeleton (written in episode 33) needs to be replaced with actor-based suite and test class runners.
2011-08-21
Let's Code Dimdwarf #64: Asynchronous State Machine
When a collaborator can work both synchronously and asynchronously, it complicates things as the caller needs to make less assumptions. In our unit tests the authenticator is a synchronous fake (it keeps the tests simpler), but the production authenticator is asynchronous (because it does blocking operations). The state machine needs to be updated to work with both synchronous and asynchronous collaborators.
2011-08-15
Let's Code Jumi #46: Multi-Interface Actors (Part 1)
After starting to write the new actors, there comes the need for actors with multiple interfaces - or multiple actors in the same thread, however you want to think about it.
2011-08-14
Let's Code Dimdwarf #63: You Will Be Assimilated
The ClientSessions class will absorb most of the logic from NetworkController, leaving the controller just wiring things together and delegating all work to others. Let's first move the authentication logic.
2011-08-07
Let's Code Jumi #41: Dynamic Events (Part 1)
I had a short up-front design session to figure out what actors the system will need. Many new interfaces will be needed for actor-to-actor communication and the current way of writing the event classes by hand will not scale. I will at first write a reflection-based solution for producing those events, because I can do that faster than it would take to figure out how to do compile-time code generation.
2011-08-06
Let's Code Dimdwarf #62: Refactoring Domino
It's curious how sometimes with TDD the refactoring steps follow each other almost automatically, like dominoes. You make one small change and then just keep on fixing test failures until it all works again and the refactoring is done.
2011-07-30
Let's Code Jumi #38: Bug Fixes
A bug in the Shade Plugin's configuration had slipped unnoticed for quite some time, so let's improve the build tests to be stricter about the contents of the build artifacts. Also a bug fix and some improvements to the TestId.compareTo() method.
2011-07-26
Let's Code Jumi #35: Actors (Part 1)
Though it was possible to make the previous end-to-end test pass without an actor model, it's in a desperate need of one, so now would be a good time to create it.
2011-07-25
Let's Code Jumi #34: Comparable TestId
Some simple refactorings before we start implementing the actor model. Using immutable collections from Functional Java simplifies some code and requires TestId to implement Comparable.
2011-07-24
Let's Code Dimdwarf #61: State Machine
The code starts to be ripe enough to be refactored into a state machine.
2011-07-18
Let's Code Jumi #30: Running One Test (Part 1)
Using the previously written public API to write the minimal testing framework, until it can report tests which it finds. Then making the end-to-end test of running one test pass with the simplest possible implementation.
2011-07-17
Let's Code Dimdwarf #60: Smells Like State Machine
As disconnecting sessions comes to play, the code starts to inhibit code smells - conditional logic. It would be possible to get rid of those smells with a state machine and the state pattern, but the smells are still quite faint, so let's implement still one more step before refactoring, so that it will be obvious that what shape the code should take.
2011-07-07
Let's Code Jumi #22: Public API (Part 1)
We continue with an end-to-end test for executing one passing test. This requires writing a minimal testing framework for executing the dummy tests, which in turn drives us to create most of Jumi's public API.
2011-07-06
Let's Code Dimdwarf #59: Session ID Mapping
When you would like to expose some internal state of a class just to be able to write tests for it, it's usually better to extract that internal thing into its own class which can then be tested through its public interface. Thus I will start writing a new class for keeping track of the mapping between sessions and session IDs.
2011-06-30
Let's Code Jumi #16: Message Passing (Part 1)
After getting the launcher and daemon to communicate with each other, it's time to refactor the system until its design fits our current understanding of the best possible design. Right now it's possible to see that some message passing infrastructure is clearly needed, so that shall be the primary focus of improvement.
2011-06-27
Let's Code Jumi #13: Running Zero Tests (Part 1)
Creating a walking skeleton to get the project started. We will start with an end-to-end test of running an empty test suite, which will drive the design for communicating with the daemon process.
2011-06-15
Let's Code Dimdwarf #58: Session IDs
Once we have timestamps, generating session IDs is trivial (but anyways worth writing a test for). Then the real thing is mapping those IDs to sessions, which is far from trivial.
2011-06-10
Let's Code Jumi #1: Project Setup (Part 1)
Jumi is my latest side project and it will be a test runner for the JVM. Its short-term goals are to run tests faster (via parallelism, class loader caching, heuristics and stuff), provide a better user experience than existing test runners, and to overcome limitations of JUnit's test runner which cause an impedance mismatch with for example most of Scala's testing frameworks. Jumi's long-term goal is to be supported by every build tool and IDE, and to surpass JUnit as the de facto test runner on the JVM.
I'll be documenting the development of this app as screencasts also for the purpose of being some research material (I know some researchers at the University of Helsinki who are interested in it). The resolution is higher (1920x1200 vs. 1440x1080) and font smaller (14 vs. 16) than in my previous videos, to make it more practical for me to do programming. Also I will try to optimize my release process so that I can release episodes faster with less effort. For example I will minimize post-production, and write a new name and description only for each theme instead of each episode.
Here is the first episode. Enjoy!
Project Setup (Part 1)
Creating a multi-module Maven project structure and using end-to-end tests to test-drive the build configuration, especially some unusual build requirements (i.e. putting a JAR in a JAR and hiding external dependencies).
2011-06-09
Let's Code Dimdwarf #57: Thread-Safe Clock
The factory for generating unique timestamps must be thread-safe. So I will write a test for thread-safeness and start using an AtomicReference.
2011-06-02
2011-05-29
Let's Code Dimdwarf #55: Pretty Printing
It's always good to be extra careful when dealing with the maximum and minimum values of an integer. After solving that bug in the tests, I'll improve the toString method.
2011-05-19
Let's Code Dimdwarf #54: Timestamp Ordering
To be able to implement session IDs, guaranteed unique values will be needed - also in a clustered environment. That can be solved by creating unique timestamps based on the paper Time, Clocks, and the Ordering of Events in a Distributed System (though first I'll implement simpler timestamps). I'll start by implementing the ordering of the timestamps, for which parameterized tests are useful.
2011-05-14
Let's Code Dimdwarf #53: Fixing Encapsulation
I will start deepening and widening the design by first fixing the encapsulation issue with the network layer - the NetworkActor's private message queue should only be accessed by the NetworkController.
2011-05-12
Let's Code Dimdwarf #52: Progressive Deepening
In this episode I get the session message roundtrip working with the simplest possible implementation - all in one method, without any layering. Of course this cannot work in production, but it lets us get started with something that works. After that it's time to do progressive deepening - introduce more layers into the software and push the functionality to the layer where it belongs to - and also do progressive widening - flesh out those layers with more functionality and production-ready quality (see Craftsmanship and Ethics at 18-21 min).
2011-05-08
Let's Code Dimdwarf #51: Walking Skeleton
My strategy for implementing session messages - and the half a dozen still nonexistent components which it depends on - is to start with a walking skeleton. This is the approach recommended in the GOOS book which I wrote about in Design for Integrability. In this episode I will explain my strategy and implement the outline of the walking skeleton.
Actually this is the second time during this project that I've used a walking skeleton. The first walking skeleton included receiving a login request and responding to it with a login failure. The first skeleton let me put in place these components: system startup, application loading, networking, authentication and the controller-actor architecture. This second skeleton will produce: unique timestamps, session IDs, worker thread handling, application APIs to listen for and send session messages, executing tasks in application code, sending session messages and committing transactions.
P.S. I wonder whether these may technically be called the first and second walking skeleton, since they don't include all main architectural components. This system has lots of essential complexity, so I prefer taking as small steps as possible. You could say that at first I implemented the legs of the walking skeleton (the controller-actor achitecture), now I implement the hands of the walking skeleton (task execution), and later I will implement the head of the walking skeleton (persistence).
2011-05-04
2011-04-30
Let's Code Dimdwarf #50: Decoding Session Messages
The end-to-end tests proved to be useful in catching a mistake in understanding the requirements which caused the unit tests to specify the wrong kind of encoding for session messages.
2011-04-28
Let's Code Dimdwarf #49: Session Messages Roadmap
Here starts the work on implementing the sending and receiving of session messages between the client and server. Implementing that will drive the design of the overall architecture quite far, because it will among other things require executing application code in worker threads and committing transactions. As usual, we start with a failing end-to-end test.
2011-04-24
Let's Code Dimdwarf #48: Command Line Parsing
I begin using the args4j library for parsing command line arguments. The tests I write are mostly learning tests.
2011-04-10
Let's Code Dimdwarf #47: Manual DI vs. Guice
The spike for manual dependency injection configuration is nearing its end and it's a good time to compare it with the original Guice-specific code which was needed to do the same thing. The infrastructure code is 50 lines of Scala for manual DI, vs. over 120 lines of Java code for Guice-based DI (which also includes lots of complex reflection). There is slightly more configuration code with manual DI, but it's also more explicit and it's easier to find out what is happening.
After this episode I did still some more refactoring off-camera, in order to reduce the duplication/boilerplate. Specifically, I created an abstract base class for the modules and passed the ServerBuilder instance to them as a hidden parameter (via a ThreadLocal), in addition to which I resolved the contention between ControllerHub and regular actors. The final code is here.
2011-04-04
Let's Code Dimdwarf #46: Encapsulated API
I move the generic code into its own classes and the DI configuration becomes quite orderly.
2011-03-24
Let's Code Dimdwarf #44: Making the Dependencies Explicit
The main principle of dependency injection is to make all dependencies of a module explicit. By refactoring the hard coded DI configuration so that the dependencies between modules become more explicit and the duplication more obvious, the design desired by the code begins to emerge.
2011-03-17
Let's Code Dimdwarf #43: It's Alive!
Now that a hard coded manual dependency injection configuration is working, it will be possible to begin refactoring it and remove all duplication, until a reusable framework or DSL emerges.
2011-03-13
Let's Code Dimdwarf #42: Without DI Frameworks
I had an idea to see how the program would fare without a Dependency Injection (DI) framework such as Guice. It should be possible to use just manual DI. It might even be much simpler that way. To find out, I will do a spike.
2011-03-09
Let's Code Dimdwarf #41: Class Loader Trouble
When the class loader reads something from a JAR file, it will lock that JAR file so that it cannot be deleted, and there is no way to explicitly dispose of a ClassLoader. Only in Java 7 will there be a solution (see the ApplicationLoadingSpec after some changes I did off-screen).
2011-03-06
2011-03-01
Let's Code Dimdwarf #40: Better Asserts
Creating your own assert methods is oftentimes useful, to make the test code and the assertion messages read better. In this episode I will also write a test for class loading from JAR files.
2011-02-23
Let's Code Dimdwarf #39: More Test Cases
It's time to write tests for the error cases. Because Specsy allows unlimited nested tests (0..*), adding multiple test fixture setups is easy - surround the existing tests with a new test block and you have a new test setup for just those tests. Even I was surprised by how easy it is.
P.S. Doesn't (0..*) look like a smiliey to you? :D
2011-02-19
Let's Code Dimdwarf #38: More Refactoring
Still some more cleaning up of the production and test code until they are tidy enough for us to move forward.
2011-02-15
Let's Code Dimdwarf #37: Refactoring
The methods were easy to move to another class when they all were static, but extracting the class does not end there - the design needs to be made right.
2011-02-11
Let's Code Dimdwarf #36: Safety Net
I will start by writing one unit test to cover the application loading, after which it will be safe to extract the code into its own class.
2011-02-07
Let's Code Dimdwarf #35: Test-After
I begin writing tests for application class loading. I had planned on test-driving some new code, but because I so seldom write code test-after, instead I decided to refactor the existing code and make it testable as if dealing with legacy code.
2011-02-03
Let's Code Dimdwarf #34: Unique Temporary Directories
When creating temporary directories in tests, a timestamp and a global variable is not unique enough. You should always assume that the tests will be run in parallel. Even when the test runner you are using does not have built-in support for running them in parallel (though it could in the future), it's possible to run the same test suite multiple times in different processes (which may happen especially on CI servers - I once had tests conflicting because of listening on a fixed network port number).
2011-01-31
Let's Code Dimdwarf #33: Extracting a Sandbox
We need a reusable utility class for managing temporary directories, so we will extract one from our existing test utilities.
2011-01-28
Let's Code Dimdwarf #32: Application Loading Test Names
Some areas of the system are still only covered by the end-to-end tests, so they need some unit tests to enable testing them faster and to put more design pressure on them. One of these areas is application loading. I begin by choosing the names of the tests to be written, which is easy because I already know based on the existing implementation that what features it has.
2011-01-24
Let's Code Dimdwarf #31: Generics Magic
I did some work off-camera to make the actors and message queues type parameterized, so that a similar bug as was in episode 25 would not happen ever again. In the beginning of this episode I will explain what I did and what kinds of Java generics tricks I was able to use to minimize the boilerplate in the Guice modules. Then it's time to write unit tests for things which are covered only by end-to-end tests.
2011-01-19
RECAP: Let's Code Dimdwarf #21-30
Recap of episodes 21 to 30, at 10 times the normal speed.
P.S. I just got a decent microphone. The sound is now much clearer than what it used to be. Unfortunately it will take until episode 42 for the regular episodes to be recorded with this mic.
2011-01-08
Let's Code Dimdwarf #30: Thoughts on YAGNI
Support for multiple clients has now been implemented. When making this system, initially I made it work on just one client, because I knew that adding features later to a simple codebase is easy and then your understanding of the problem will be better. It took only a bit over 4 hours to add support for more than one client - surely not much longer than if I had built it in already in the beginning.
2011-01-05
Let's Code Dimdwarf #29: Authenticating Multiple Clients
The last thing remaining to support multiple clients is to update the authentication layer.
Subscribe to:
Posts (Atom)