Testing Trello Webhooks With Ngrok

1. Introduction

In this article, we are going to have a look at an example of testing Trello webhooks with Ngrok.

Testing a webhook locally is a challenging task as the localhost is not accessible over the internet and the webhooks require a URL that can be accessed outside of your network/firewall.

In this easy-to-understand tutorial, we’ll have a look at how to test the webhooks locally and easily, without much hassle.

2. What is a Webhook?

A webhook is an HTTP callback. You create a register a webhook with some external application by providing a callback URL. When something happens, the external application will send an HTTP request to the callback URL.

In this use case, we are using Trello. Whenever someone creates a new Card on Trello, we want to save that information to a file.

To achieve this, we need some way to integrate with Trello and we need some way using with Trello will notify us that a new Card has been created. Webhooks solves this problem.

We’re going to register a webhook with Trello and give it a callback URL. Whenever a new Card is created, Trello will send an HTTP POST request to the callback URL. The POST body will contain all the details of the newly created Card.

3. Register a Webhook

In order to register a Webhook, we’ll need Trello API Key and Token. You can grab yours here. Once you have the Key and Token, we need a callback URL.

In your web application, create a new endpoint in this example, we’ll create one new endpoint http://localhost:7070/webhooks/board.

We’ll expose this endpoint over HTTP GET as well as HTTP POST. The GET method is required because when we register a Webhook, Trello will do a quick check to see if it gets a 200 OK response.

The POST method is the one where we’ll do our actual work.

The Webhook registration request looks like this:

HTTP POST https://api.trello.com/1/tokens/{APIToken}/webhooks

{
    "key": "{APIKey}",
    "callbackURL": "http://localhost:7070/webhooks/board",
    "idModel":"4d5ea62fd76aa1136000000c",
    "description": "My first webhook" 
}

One thing to note here is that we cannot directly use http://localhost:7070/webhooks/board in the Webhook registration request. The reason is that the localhost is not accessible by the Trello API.

We need a mechanism to expose the localhost over the internet. Here Ngrok comes into the picture.

The idModel is the id of the Trello Board that we need to watch and its not a Webhook concern. Refer Trello documentation for more.

3.1 Exposing localhost Using Ngrok

Ngrok creates a tunnel to your localhost and serves it over a public URL.

  1. Download Ngrok from here.
  2. Extract it to your machine.
  3. Start your web server on any port, say 7070.
  4. Run ngrokhttp 7070. Ngrok will start now.
  5. Go to http://localhost:4040. This is the Ngrok UI.

Ngrok

On the Ngrok UI, you will see something like this. The URL http://78d81a9b.ngrok.io will tunnel to http://localhost:7070. This means, http://78d81a9b.ngrok.io/webhooks/board tunnels to http://localhost:7070/webhooks/board.

This solves our problem. Thus, we can simply replace localhost URL with this public URL in the call to our Webhook.

Once done, whenever we create a new Card, we’ll get HTTP POST request on http://localhost:7070/webhooks/board, which will then be forwarded to our application on localhost.

So, our updated request looks like this:

HTTP POST https://api.trello.com/1/tokens/{APIToken}/webhooks/
{
  "key": "{APIKey}",
  "callbackURL": "http://78d81a9b.ngrok.io/webhooks/board",
  "idModel":"4d5ea62fd76aa1136000000c",
  "description": "My first webhook"  
}

Fire the request and create a new Card on your Trello board. Once done, your Ngrok UI will be updated with the new request:

Ngork Post request

This request should be transferred to our web application running on the local host.

If we want to debug the same request again, simply click Replay button on the UI.

4. Conclusion

In this article, we learnt that the testing Webhooks locally poses a unique challenge. We also learnt that using Ngrok makes it pretty easy to test the Webhook callbacks.

The complete source code from this article can be found over on GutHub.

Here is the video demo.

TDD in Action

This article is a continuation of my previous article where I discussed TDD methodology. In this article, we are going to see TDD in action.

I thought it’s better to demonstrate TDD through a video instead of writing an article. So, here is the video that I recorded to demonstrate the use cases. I hope you will enjoy it and learn something new. Please let me know in comments.

The full source code is available on GitHub.

 

Introduction to TDD

When you realize the true power of tests for the first time

Introduction

TDD or Test Driven Development is a development methodology that completely changes the way we develop our code. Although this has drawn some criticism, its still getting more and more popular and TTD is considered as an important skill in many job listings.

TDD forces the developers to write the tests first, even before writing the production code. Wierd, you say. Not really, actually.

Why Tests First?

Why not? I mean, don’t treat your tests as second-class citizens of your project. They are as important as the production code, if not more.

Tests are your weapon, the stronger they are, the more powerful you become. They enable us to understand the code, quickly modify it, and bring down the maintenance cost. Don’t believe me? Take my word for it.

What is the best way to understand any new library or a framework? Even we have well-written docs in good English, what we, as developers, always look for is the “code examples“. This is because we prefer well-written code over well-written sentences.

Our tests are code examples of our production code. The tests tell other developers how to use this code and what scenarios should they consider. This brings down the learning curve.

Test Driven Development

How to do TDD?

In TDD, we start with tests first. Since we are starting with tests, it’s obvious that the test will fail. Then we write the production code just enough to pass the failing test. Once done, write another failing test and the process repeats. This way, we are writing code in very short test/development cycles.

It makes sure that all of our production code is covered with the unit tests, which is a good thing.

Following this process forces the developers to have clear API contracts. They are forced to think how the methods are gonna be used by the outside world. When I say API, I mean any public methods that we expose from our classes.

It automatically makes our code easily testable.

Many would argue against this practice saying that it requires more time, but actually, it doesn’t. Whenever we feel like making any changes to the code, we will be able to do so effortlessly, knowing that the tests have our back. This will save a lot of time. Always remember, the code is going to change, today or tomorrow. And when the time comes, you will bless yourself or the original developer for writing such concrete tests.

Fragile Tests

Ever been in a situation where you modify a small bit of code that results in multiple test failures and you find yourself modifying tiny pieces of code in all the failing tests just to find some other tests failing?

Congratulations, you have discovered fragile tests. Many people find themselves stuck in the loop of fragile tests while following the TDD. But think again, is it really the problem of TDD? Or an advantage of TDD?

Having fragile tests that break every now and then is not the problem with TDD, in fact, its the problem with the tests themselves. TDD or not, bad tests are bad tests. They will keep haunting the developers till they exist.

So how does TDD help in writing effective tests? If you find yourself modifying every test while writing code, it is a signal that our tests need refactoring. It’s better to refactor 10 tests than refactoring 50 tests.

BUT…

But my teammates are not interested in TDD. But my manager doesn’t approve of TDD. But my architect thinks TDD we are not prepared for TDD. But, but, but…

This will never end. Understand that explaining the benefits to the team is our responsibility. We need to make them understand that this is the correct way. Even then if no one listens to you, we can still follow the practice on our own.

It is up to the individual whether to follow TDD. Our managers need not worry about the methodology if your performance is good. It’s an individual’s practice.

The next article/video shows you how to actually write code using TDD methodology. See it in action.

If you are just starting with Java or are confused with terms like Java SE and Core Java, head on to this article to simplify things.

Starting Java Development Career

Starting a Java development career can be a great choice given the number of job openings that pops up day after day. Java has been around for a while and there are a tremendous amount of companies that already have a Java system in place. This only means a lot of development opportunities.

Java is compiled, type-safe, and fast. I mean really fast. I have myself migrated a fairly complex project from Python to Java and another one extremely complex system from .Net to Java. The main reasons were performance improvement and platform independence.

Learning Java in itself is not that difficult, but in order to become job-ready and be able to develop real-world Java applications requires a fair amount of knowledge. In this post, we’re gonna list down the skills required for becoming a job-ready Java developer.

Core Java

Core Java or Java SE is the starting point of your Java development career. Many beginners think that once they learn Java SE, they are ready to develop real-world applications. This is not true. First of all, in order to be a good programmer, learning the syntax is not enough. You should learn how to design an application by using various constructs provided by the programming language. You need to master it by writing a lot of code and developing some projects keeping the design principles in mind.

Next thing is that Java SE itself is not enough. A quite common application would be to develop a command line application which has limited use. To get to the next level, you need to learn Java EE or Advanced Java.

Advanced Java

Java EE or Advanced Java supports the development of server-side backend web applications. While an understanding of Java EE concepts like servlets, JSPs, and EJBs is useful, you need not master this. Just the basic level of knowledge should be sufficient.

The main reason for this is that developing any web application by simply using Java EE is difficult and time-consuming. There are a lot of frameworks available today that help in this development. Some of them are Spring, Spark Java, and Dropwizard.

These frameworks make web application development so easy that you don’t even need to know about JSP and Servlets in order to develop a basic web application.

If you are not sure which framework you should use, go for Spring. Its the most widely used framework and has a very active community.

Testing

Testing is an integral part of Software development. Writing automated unit tests helps developer receive instant feedback on the code. Unfortunately, many developers consider tests as second-class citizens.

The de-facto for unit tests in Java is Junit. Junit is widely used and has a great community out there. I can’t stress less on the importance of tests. You just need to master it.

Tools

Before you start with any framework, you must be familiar with some build tools and IDE that makes your job easy.

Build Tools

A Java application can consist of lots and lots of files and one big project might be divided into smaller modules where one module is dependent on another. Apart from this, your code might be using some popular libraries like apache commons to solve some well-known problems that come with every project.

These dependencies need to be supplied from your development environment to test environment and then to the production. This can become a daunting task if done manually. To solve this problem, we use build tools like Maven or Gradle.

If you are unsure of what to start with, choose Maven. It’s the older among the two, has wide applications, good community support, and easier to begin with.

IDE

An IDE or Integrated Development Environment helps you a lot. It saves you a lot of headache by providing the features like real-time compilation – see the compilation errors as you type, no need to wait till actual compilation happens. An IDE provides type completion among other things.

Code completion
Code completion example

There are a lot of IDEs available but only two of them are really good. My preference is IntelliJ Idea and then Eclipse.

I use Ultimate edition of IntelliJ which is paid, but there is Community edition which is free. Eclipse is also a free IDE. There is another IDE called STS (Spring Tool Suite) which is essentially Eclipse with plugins for Spring development.

Git

Git is a distributed VCS (version control system). Every project uses one or the other VCS for source code management. Git keeps different versions of the source code files as you modify them. Making tracking changes and code history a manageable task.

Server

A web application needs to be deployed on a server. There are several choices like Tomcat, Jetty, GlassFish. A beginner should start with Tomcat. It’s really lightweight and very easy to configure and use. However, if you are going to develop your first web application using Spring Boot, then there is no need to worry about servers for a small project. Spring Boot allows you to use an embedded server.

Conclusion

In essence, if you want to get a job as a Java developer, you need to learn about Java Development along with the most popular frameworks and tools. Using a popular framework will increase your chances of employability as well as it will be easier for you to learn.

So once you get hold of Java SE, develop some small projects to get hold of the language. Then Start learning Spring Boot + Maven and use STS for development. Once you are familiar, develop more and more web applications.

You might want to check out the resume polishing tips before applying for a job. All the best.

Polishing Your Resume Before Applying For a New Job

A resume is your first impression on your prospective employer. You better spend quality time on polishing your resume before applying for a new job. In this article, we will discuss some of the tips and tricks that I have found to be extremely useful in getting your resume shortlisted.

Avoid Walk-ins and Bulk Hiring Processes

I know, it might sound a bit counter-intuitive. But once I have got some experience, I apply for jobs that have some specific and clear requirements. In that case, I will be able to know the type of project and the future team that I will work on during the interview process.

Generally, walk-ins do not tend to better judge the potential candidate and usually do not give the candidates a better chance to evaluate the job. The hiring companies do not generally have a clear vision about the deployment of resources. And most importantly, you lose bargaining power in terms of salary negotiations.

Of course, all this is true for service-based organizations. If its the (rare) case of a product based company, things change. Product based companies generally have a single product or a fixed set of products on which the new talent will be deployed. You will most likely end up in the team of the interviewer. And product based companies usually pay better than their service-based counterparts.

Now all this is just my general observation and experience. The real world is filled with randomness and nothing can be confined to a set of well-written rules. So expect the unexpected.

Target a Company and Do Some Research

Whenever I am looking for a job change, I select a specific company where I will apply and do some research to understand the work culture and type of job that I might get.

I don’t apply for every job opening that I find on job search sites. Instead, I take time to learn about the company and shortlist a few of them that fits my parameters. Then, I apply to one or two of the top companies that I have shortlisted and I give everything I have to get in.

My research starts with the job description itself. The type of tools and technologies that are required, the roles and responsibilities listed, etc. Then I do a general google search to see what people are talking about the company. Sometimes you might also get to know if the customers are happy with it. Then comes glassdoor, where you can find the salary range and the reviews from the current and past employees. I read a lot of reviews. Then I head towards the interview section where you can have a peek inside the interview process and what types of questions you might face and what are the experiences of the past interviewers.

Then lastly, visit the company’s official website to learn about the management and the type of work they are in to. All this helps in shortlisting the company. I look for negative points like frustrated customers/employees, technological skillset mismatch, extremely lower salaries/increments. If I find more of such negative flags, I become cautious.

Polish Your Resume, For Every Job That You Apply

Yes, that’s right. I don’t use a common resume for all the jobs. The reason is simple, not all jobs have the same requirement.

Circles of knowledge

The left circle is the circle of your knowledge that you have built up based on your research. This circle tells you about the types of requirements and qualities that are most important to this particular job. This could be anything ranging from experience with a specific tool or technology, experience working on a certain type of project, leadership skills, or anything else.

The right circle is the current skill set that you have. This is fixed and can’t be changed over a very short time.

The intersection of these two circles makes you a better or worse fit for any given job. I highlight the portion of my resume that falls at the intersection point.

For example, at one of the jobs that I applied, I got to know that they put a lot of emphasis on knowledge of design patterns and also testing.

I highlighted the part that talked about unit testing and described how I created a base testing framework in one of the projects in the past. Also, in the project description section, I included a row mentioning the design patterns that we had used.

Result? The recruiter said that I am a “great fit” for the job.

Now it should be obvious that why I don’t use the same resume for all the applications. The right circle changes depending on the job. Hence, the section of intersection also keeps on changing.

High-level Structure of a Resume

Even though the contents of my resume keeps on changing, I have designed a concrete high-level structure that remains the same. This allows me to do quick edits and customize the resume quickly.

The first section consists of an introduction and a profile summary. The introduction is usually 2-3 sentences and the profile summary consists of 7-10 bullet points that highlight what I have achieved in my past jobs. Note this, “highlight what I have achieved” and not “what I have done”. I like to represent myself as an “achiever” rather than a “doer”.

Achievements could be anything from helping clients meet a deadline in a highly ambitious timeframe, achieving performance improvements, improving quality among others.

Next comes the experience section that describes in detail my roles and responsibilities in past organizations along with the project details.

Then come the section for academics and professional certificates. A really short section that does not take much of space. Only consist of the highest educational degree.

Lastly, there are public artefacts that contain links to my blog, GitHub profile, and other things that I find relevant.

And yes, all this fits really well in two pages. The interviewer immediately loses interests if the resume has more than two pages. Extra pages should be included only when it is absolutely required and when you have a lot of experience. I never went beyond two.

Things that I avoid in resume are the team size in project details and my hobbies. Its just a matter of personal preference. No one ever asked me why hobbies are missing in my resume.

How to Love Your Code

1. Introduction

In this article, I am going to present my strong personal opinions on various aspects of the code, legacy applications, and why that existing ugly looking method does not use generics? Feel free to agree, disagree, and provide any constructive feedback.

2. The Problem With Legacy Code

The main problem with the legacy code is that it isn’t the problem at all. That ugly looking method that heavily relies on HashMap was written in Java 1.4, when no had heard about the fancy word Generics. Before cursing the existing code for not being readable and not doing what we want, try to understand the circumstances under which it was written.The code might not do what we want, but hey, this the exact code that is running on production and earning $$ for the company. It is doing exactly what it’s supposed to do. It is the code that the customers are using day in day out. This is the code that generating the revenue and even our salaries.

The code might not do what we want, but hey, this the exact code that is running on production and earning $$ for the company. It is doing exactly what it’s supposed to do. It is the code that the customers are using day in day out. This is the code that generating the revenue and even our salaries.

Now, don’t think I am promoting spaghetti and unstructured code in any way. My point is, if we have a problem, then we should carefully look at it, observe it, and then try to think of ways to resolve it. We can’t just curse it and sit with hand on our hand.

Many times it happens that when I look at the code that I wrote six months back, I immediately think, did I write this crap? I am sure there will be many who agrees to this. If we can’t appreciate our own code that was written a few months back, how can we blame the code that was written 2, 5, or even 10 years back. Believe me, that code was “as per the current industry standards” while it was written.

3. Stop Complaining, Start Acting

So, what should be done in this case? Improve the code. But how?

Utmost care should be taken while refactoring the legacy code. And, by the way, there is a reason it’s called “legacy code”, because it has written a legacy (I think this one is from  Joel Spolsky).

Completely rewriting the entire code base is almost never a good idea. However, refactoring it bit-by-bit is the approach that we should embrace.

In our team, first, all of the team members agreed that we need to improve the existing code. Then, we identified some of the ways that we can achieve this, some automated, some manual. We took several pledges to reach our goal.

Pledge 1: Clean the Surroundings

First, we decided to start cleaning the surrounding area of code where are working. Remove the dead code and commented code, add generics to that ugly looking method, extract common code in separate methods to make it DRY, and many other small things. This started building the confidence in our team. Such things are very easy to master if you can make good use of your IDE.

Pledge 2: Strict Code Reviews

We decided that we will commit the code without peer reviews. This helped us to stay on track and avoid the temptation of “cheating”. This also made sure that everyone on the team is committed towards our common goal. This went for a while until everyone developed confidence in everyone else.

Pledge 3: More and More Junits

This is the best thing that we decided to do. Writing Junits, mocking, spying, stubbing, these quickly became the buzz words. Writing automated test cases gave us tremendous confidence to refactor the existing code with even more speed. I personally decided to never compromise on automated tests after this experience.

Pledge 4: No Sonar Criticals/Blockers

We integrated SonarQube with our project. The results of the first scan were really disappointing, but we didn’t lose hope. We worked hard to resolve the Blockers and Criticals until there were zero occurrences. Later, we started our nightly builds and integrated SonarQube with it. Every morning, we use to verify if the code committed yesterday is having any high priority issues. Gradually, we started focusing the Major issues.

This helped us reduce code smells and code complexity.

4. Well Done Is Better Than Well Said

I admit that it’s not as easy as it might sound. It took us almost a year before everyone started appreciating the code. Our clients thanked us, our managers appreciated us, and everyone was more confident than ever. But this has not yet stopped. This has to continue. There are still several areas of improvements. This is a “legacy”.

Replacing Conditional Getters With Functional Programming in Java

1. Introduction

In this small article, we will have a look at how to efficiently write a code that runs a getter (or any other method) based on some input using functional programming.

2. Problem

Instead of explaining it in words, let me show you the code directly:

private int normalMethod(MyPojo pojo, Actions action) {
    if (action == Actions.ACTION1) {
        return pojo.getActionOne();
    } else if (action == Actions.ACTION2) {
        return pojo.getActionTwo();
    } else if (action == Actions.ACTION3) {
        return pojo.getActionThree();
    } else {
        return pojo.getActionFour();
    }
}

As we can see, we are calling the appropriate getter based on the Action that we get in the input. This seems to be pretty decent at the moment, but as and when there are new actions/getters, we need to modify the code and the chain can grow longer.

3. Solution

One good solution to this is to make use of functional programming in Java. For this, we need to create a mapping of the Actions to method references:

public static final Map<Actions, Function<MyPojo, Integer>> GETTERS_MAP = new HashMap<>();
GETTERS_MAP.put(Actions.ACTION1, MyPojo::getActionOne);
GETTERS_MAP.put(Actions.ACTION2, MyPojo::getActionTwo);
GETTERS_MAP.put(Actions.ACTION3, MyPojo::getActionThree);
GETTERS_MAP.put(Actions.ACTION4, MyPojo::getActionFour);

As we can see, we have mapped every Action to the corresponding getters. Once this is done, our long if-else chain now reduces to only a couple of lines:

private Integer functionalMethod(MyPojo pojo, Actions action) {
    Function<MyPojo, Integer> getter = MyPojo.GETTERS_MAP.get(action);
    return getter.apply(pojo);
}

And that’s it. Even if we need to add another Actions/getters, we just need to add an entry in the Map. The method above doesn’t need to be modified.

4. Conclusion

We have observed that functional programming opens new doors of creativity. Please find the full source code of this article on GitHub.

Actuators in Spring Boot

It is really important to be able to monitor various metrics of a production application. One way to achieve this is to use the Actuators in Spring Boot.

1. Configuration

Like many other things, the configuration of Spring Boot Actuators is really simple. Just add the below dependency in your application.

Gradle

[code language=”sql”]
compile group: ‘org.springframework.boot’, name: ‘spring-boot-starter-actuator’, version: ‘1.5.1.RELEASE’
[/code]

Maven

[code language=”xml”]
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>1.4.2.RELEASE</version>
</dependency>
[/code]
Continue reading “Actuators in Spring Boot”

Creating a strong foundation for a new project using Spring Boot – Part 4

Contents

This post is divided into four parts.

  1. Getting Ready
  2. Unit testing
  3. Integration testing
  4. Launching the app

Launching the app

At this point, our application is complete. You can launch the app by executing the following command.

[code language=”plain”]

gradlew bootRun

[/code]

Make sure you have modified your application.properties according to your DB and that your DB is up and running.

Try our some GET/POST requests. You can use CURL or a GUI based chrome extension Postman. There are similar tools available for other platforms. Make sure to query your DB to check the modifications.

Adding the ability to run anywhere

Making your application flexible enough to be able to run on anywhere could be very useful. Particularly when you have to demonstrate. Continue reading “Creating a strong foundation for a new project using Spring Boot – Part 4”

Creating a strong foundation for a new project using Spring Boot – Part 3

Contents

This post is divided into four parts.

  1. Getting Ready
  2. Unit testing
  3. Integration testing
  4. Launching the app

Integration tests

In integration tests, multiple layers communicate with each other and we test multiple layers of the system in a given test. It’s a good idea to keep your integration tests separate from unit tests. You will run unit tests more often, with every build for example. The integration tests might run with a nightly build.

Create two new source folders src/integrationTest/java and src/integrationTest/resources in your IDE. In eclipse, Right click on project>New>Source folder.

Create a new file application-integrationtest.properties in  src/integrationTest/resources. Continue reading “Creating a strong foundation for a new project using Spring Boot – Part 3”