Radaxian radio: la mujer en los videojuegos

Esta semana mi colega Frankkylin me invitó a participar en su podcast Radaxian Radio, para debatir sobre la situación hoy en día de la mujer en el mundo de los videojuegos. Sin más os dejo el enlace para que escuchéis a Frankkylin, Raul Cautivo, Julio J Gamez, Ruth, SaraNina.

Artwork de Radaxian radio: la mujer en los videojuegos

Ir a descargar

Anuncios

Booting a REST API with swagger

Generating an easy-to-use documented REST API, and run it on-the-fly

Publicado en CodingMarkers el 28 de marzo del 2018

Booting a REST API with swagger

One of the most usual tasks requested for back-end developers is generating a microservices oriented API. When we talk about microservices, we refer to small sets of endpoints, each one of them resolving a different problem, which can be deployed individually in a fast and simple way, so they are easy to escalate in case we need to take care of high amount of requests in a certain area. This is a great advantage compared to the classic monolithic architecture, which requires the deployment of all the services at once, taking more time and resources.

REST services are more efficient than SOAP services, and also easier to securize, so it’s no surprise they are becoming so popular, but deploying and testing the services can be troublesome. Fortunately there are some frameworks which provide us some easy-to-use tools.

Let’s split this issue in 3 steps:

  1. Generating the REST service using JSON.
  2. Booting the service with Spring boot.
  3. Document it and get a user-friendly interface for tests with Swagger.

In our example code, we are going to simulate a small system to handle a videogame database via REST. The code packages will be defined as follows:

  • The API package will contain all the REST API, dividing it by layers (services, data access and data transfer objects), and providing the controller with the endpoints.
  • To set up the swagger system we will need a configuration file, plus the API package classes should get new annotations to document its content.
  • Finally we will add the generic Spring boot runner, which may be used for any package and requires very little tuning.
The hierarchical packages structure
The hierarchical packages structure

 

1. Designing a dummy RESTful web service

The first step is creating a basic structure for the REST service. We’ll split it up in different layers:

  • The controller would be VideogameController, and would contain all the endpoints, we’ll leave this for later as, we’ll talk about setting it up with swagger.
  • The interface IVideogameService will define the service layer, which would have the “normal” endpoints.
  • The IVideogameDAO gives an idea of the methods available through the Data Access Layer. In order to void much otrouble setting up a database connection or a real repository, we will mock it.

A good practice is to split the design and the implementation by using interfaces. It’s not really necessary, but it will make the code more reusable.

The UML diagram with the layers
The UML diagram with the layers

Therefore the Service would simulate commonly known as CRUD operations: get (find), set (update), add (save) and remove (delete).

import org.thalion.snippet.swagger.api.dto.Videogame;

public interface IVideogameService {
  Collection<Videogame> findAll();
  Videogame findOne(String name);
  String save(Videogame videogame);
  void update(Videogame setName);
  void delete(String name);
}

The service won’t have much to do today, it will only connect to the data access layer and let it handle all the database work.

import java.util.Collection;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.thalion.snippet.swagger.api.dao.IVideogameDAO;
import org.thalion.snippet.swagger.api.dto.Videogame;

@Service
public class VideogameService implements IVideogameService {

  @Autowired
  private IVideogameDAO dao;

  @Override
  public Collection<Videogame> findAll() {
    return dao.findAll();
  }

  @Override
  public Videogame findOne(String name) {
    return dao.findOne(name);
  }

  @Override
  public String save(Videogame videogame) {
    return dao.save(videogame);
  }

  @Override
  public void update(Videogame videogame) {
    update(videogame);
  }

  @Override
  public void delete(String name) {
    dao.delete(name);
  }
}

The Data Access Object or DAO will actually do the hard work, it’s the one which will connect to the database.

import java.util.Collection;
import org.thalion.snippet.swagger.api.dto.Videogame;

public interface IVideogameDAO {
  Collection<Videogame> findAll();
  Videogame findOne(String name);
  String save(Videogame videogame);
  void update(Videogame videogame);
  void delete(String name);
}

But in this case we want to keep it simple since we are focusing on the services layer., so instead of configuring a database connection and writting some queries, we are going to mock it with a simple Map structure.

import java.util.Collection;
import java.util.Map;

import org.springframework.stereotype.Repository;
import org.thalion.snippet.swagger.api.dto.Videogame;

@Repository
public class VideogameDAOMocker implements IVideogameDAO {

  private Map<String, Videogame> storage;

  @Override
  public Collection<Videogame> findAll() {
    return storage.values();
  }

@Override
public Videogame findOne(String name) {
  return storage.get(name);
}

  @Override
  public String save(Videogame videogame) {
    storage.put(videogame.getName(), videogame);
    return videogame.getName();
  }

  @Override
  public void update(Videogame videogame) {
    storage.put(videogame.getName(), videogame);
  }

  @Override
  public void delete(String name) {
    storage.remove(name);
  }
}

The Data Transfer Object or DTO isn’t really interesting right now, as it would be just as simple as a class with a couple of Strings to define its attributes. So we will skip its content for now.

Finally, we will refer to the endpoint as the Controller, and since we would be using a REST service with JSON, we will need to add its maven dependency.

<dependency>
  <groupId>com.sun.jersey</groupId>
  <artifactId>jersey-json</artifactId>
</dependency>

The Controller for Spring MVC would be an extra layer where most of the interesting things we are studying today will happen. First of all, we are going to connect it with the service, and set up the REST via spring JAX-RS annotations:

  • RequestMapping configures the path.
  • RequestMapping states the method expects an HTTP request.
  • RequestBody gets the information from the HTTP body.
  • PathVariable açgets a value from the URL.
  • ResponseStatus will store the status code to send back.
  • ResponseBody gets the information from the HTTP body
import java.util.Collection;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;
import org.thalion.snippet.swagger.api.dto.Videogame;
import org.thalion.snippet.swagger.api.service.IVideogameService;

@RestController
@RequestMapping("/api/videogames")
public class VideogameControllerBeta {

  @Autowired
  private IVideogameService service;

  @ResponseStatus(HttpStatus.OK)
  @RequestMapping(method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
  public Collection<Videogame> getAllVideogames() {
    return service.findAll();
  }

  @ResponseStatus(HttpStatus.OK)
  @RequestMapping(method = RequestMethod.GET, value = "{name}", produces = MediaType.APPLICATION_JSON_VALUE)
  public Videogame getVideogameByName(@PathVariable String name) {
    return service.findOne(name);
  }

  @RequestMapping(method = RequestMethod.POST, produces = MediaType.APPLICATION_JSON_VALUE)
  public ResponseEntity<String> createVideogame(@RequestBody Videogame videogame) {
    String videogameCreatedId = service.save(videogame);
    return new ResponseEntity<String>(videogameCreatedId, HttpStatus.CREATED);
  }

  @ResponseStatus(HttpStatus.NO_CONTENT)
  @RequestMapping(method = RequestMethod.PUT, value = "{name}")
  public void updateVideogame(@PathVariable String name, @RequestBody Videogame videogame) {
    videogame.setName(name);
    service.update(videogame);
  }

  @ResponseStatus(HttpStatus.NO_CONTENT)
  @RequestMapping(method = RequestMethod.DELETE, value = "{name}")
  public void deleteInfo( @PathVariable String name) {
    service.delete(name);
  }
}

With this we are all set-up, so let’s find somewhere to run this code.

2. Let’s boot it!

Spring boot is a tool to generate a “just run” application, which can be set up with only a few lines of code an annotations, without any xml configuration. This makes it really interesting in order to run and test Java microservices really fast.

Let’s start by adding its maven dependencies and setting up its just run configuration:

<dependencies>
  <!-- spring boot setup ability -->
  <dependency>
    <groupId>org.springframework.boot>/groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
</dependencies>

<!-- Just run configuration -->
<build>
  <plugins>
    <plugin>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-maven-plugin</artifactId>
    </plugin>
  </plugins>
</build>

Then, we need to set it up in the Java code itself. The longest way to configure it would be using the following annotations:

  • RestController, it’s an Spring-MVC, which sets up this class as controller for a REST service
  • EnableAutoConfiguration, this configures the boot service according to dependencies we have included. Since we used TesController, the system will consider to add Spring MVC and Tomcat.
  • ComponentScan, scans the components of the package and its children packages.

But all this 3 annotations can be reduced to a single one: ‘SpringBootApplication (scanBasePackages = { “replace_with_main_package” })’, which is the one used in the example.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

/**
* Generic SpringBoot, only configured by setting the scanBasePackages restricts the scanned packages
*/
@SpringBootApplication(scanBasePackages = { "org.thalion.snippet.swagger" })
public class SpringbootRunner {

  public static void main(String[] args) {
SpringApplication.run(SpringbootRunner.class, args);
  }
}

So just by running this class, a local server will be deployed, and we are ready to test the service… but that’s not intuitive. Let’s make it better.

3. Making it pretty and easy to test, plus avoiding setting up the client side

Documenting an API to make it easy to understand for other developers, and make it friendly for the testers is no easy task. Swagger is a framework which, using a few extra annotations, is able to generate a simple web user interface with REST calls to the API documented, using the metadata available. We are going to use the springfox version, as it comes already bundled with the correct annotations.

❕This was the stable version when the post was originally written.

<dependencies>
  <!-- Swagger -->
  <dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger-ui</artifactId>
    <version>2.3.0</version>
  </dependency>
  <dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger2</artifactId>
    <version>2.3.0</version>
  </dependency>
</dependencies>

The most usual annotations are:

  • Api: for controller classes, it sets the API endpoint documentation.
  • Apimodel: for Data Transfer Objects class names.
  • ApiModelProperty: for Data Transfer Objects attributes.
  • ApiOperation: they go above the methods or services available on an endpoint.
  • ApiParam: for input parameter on a method.
  • ApiResponse: for output parameters. You can have more than one with a set of ApiResponses (e.g. also add the 404 error, etc).

Then, in order to fully set this up, there are 2 important points to recheck: the DTO and the controller.

Let’s start with the DTO, as it’s the most straight-forward part: we only need to document the information about the class definition itself and the attributes.

package org.thalion.snippet.swagger.api.dto;

import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;

@ApiModel(value = "Videogame entity", description = "Complete data of an entity videogame")
public class Videogame {

  @ApiModelProperty(value = "The name of the videogame", required = true)
  private String name;
  @ApiModelProperty(value = "The developer of the videogame", required = false)
  private String developer;

  public String getName() {
    return name;
  }

  public void setName(String name) {
    this.name = name;
  }

  public String getDeveloper() {
    return developer;
  }

  public void setDeveloper(String developer) {
    this.developer = developer;
  }
}

The controller is a bit tougher, as we need to document every single endpoint.

import java.util.Collection;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;
import org.thalion.snippet.swagger.api.dto.Videogame;
import org.thalion.snippet.swagger.api.service.IVideogameService;

import io.swagger.annotations.ApiOperation;
import io.swagger.annotations.ApiParam;
import io.swagger.annotations.ApiResponse;
import io.swagger.annotations.ApiResponses;

@RestController
@RequestMapping("/api/videogames")
public class VideogameController {

  @Autowired
  private IVideogameService service;

  @ResponseStatus(HttpStatus.OK)
  @RequestMapping(method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
  @ApiOperation(value = "Get Videogames", notes = "Returns all the videogame data")
  @ApiResponses({ @ApiResponse(code = 200, message = "Returns this information") })
  public Collection<Videogame> getAllVideogames() {
    return service.findAll();
  }

  @ResponseStatus(HttpStatus.OK)
  @RequestMapping(method = RequestMethod.GET, value = "{name}", produces = MediaType.APPLICATION_JSON_VALUE)
  @ApiOperation(value = "Get the info for one videogame", notes = "Returns the info from one videogame")
  @ApiResponses({ @ApiResponse(code = 200, message = "Exists this information") })
  public Videogame getVideogameByName(
@ApiParam(defaultValue = "default", value = "The name of the videogame to return") @PathVariable String name) {
    return service.findOne(name);
  }

  @RequestMapping(method = RequestMethod.POST, produces = MediaType.APPLICATION_JSON_VALUE)
  @ApiOperation(value = "Create videogame information", notes = "Create a videogame entry")
  @ApiResponses({ @ApiResponse(code = 201, message = "The videgame entry was created successfully") })
  public ResponseEntity<String> createVideogame(@RequestBody Videogame videogame) {
    String videogameCreatedId = service.save(videogame);
    return new ResponseEntity<String>(videogameCreatedId, HttpStatus.CREATED);
  }

  @ResponseStatus(HttpStatus.NO_CONTENT)
  @RequestMapping(method = RequestMethod.PUT, value = "{name}")
  @ApiOperation(value = "Update videogame information", notes = "Update a videogame information entry")
  @ApiResponses({ @ApiResponse(code = 204, message = "The videgame entry was updated successfully") })
  public void updateVideogame(
@ApiParam(defaultValue = "Default", value = "The name of the videogame to update") @PathVariable String name,
@RequestBody Videogame videogame) {
    videogame.setName(name);
    service.update(videogame);
  }

  @ResponseStatus(HttpStatus.NO_CONTENT)
  @RequestMapping(method = RequestMethod.DELETE, value = "{name}")
  @ApiOperation(value = "Delete videogame", notes = "Deletes a videogame entry")
  @ApiResponses({ @ApiResponse(code = 204, message = "The videgame entry was deleted successfully") })
  public void deleteInfo(
@ApiParam(defaultValue = "Default", value = "The name of the videogame to delete") @PathVariable String name) {
    service.delete(name);
  }
}

Once all the documentation annotations have been written down, we just need to configure the API page with the help of a Docket builder, which provides the primary graphic interface for our swagger implementation.

Since we are working with an API, we are only going to worry about the URLs under the “/api/” pattern, which we can select with the help of regular expressions.

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import springfox.documentation.builders.ApiInfoBuilder;
import springfox.documentation.builders.PathSelectors;
import springfox.documentation.service.ApiInfo;
import springfox.documentation.spi.DocumentationType;
import springfox.documentation.spring.web.plugins.Docket;
import springfox.documentation.swagger2.annotations.EnableSwagger2;

@Configuration
@EnableSwagger2
public class SwaggerConfig {

  @Bean
  public Docket newsApi() {
    return new Docket(DocumentationType.SWAGGER_2).groupName("api-videogames").apiInfo(apiVideogames()).select()
.paths(PathSelectors.regex("/api.*")).build();
  }

  private ApiInfo apiVideogames() {
    return new ApiInfoBuilder().title("Videogames REST api POC")
.description("PoC of a REST api, to test both Springboot and Swagger")
.termsOfServiceUrl("https://creativecommons.org/licenses/by/4.0/").contact("thalionte@gmail.com")
.license("GNU General Public License v3.0").licenseUrl("https://www.gnu.org/licenses/gpl-3.0.en.html")
.version("3.0").build();
  }
}

And we finally get this beautiful webpage with all the endpoints, which we can easily interact with on “localhost:8080/swagger-ui.html”. We can interact with the different methods and set up the arguments to do quick tests, avoiding the need to write a client or using a REST requests generator like SoapUI.

Congratulations! your demo is completely set up and ready to go.

The swagger page result
The swagger page result

Installing Visual Basic 6.0 in Windows 10

Back to the past

Publicado en CodingMarkers el 28 de marzo del 2018

Installing Visual Basic 6.0 in Windows 10

I was recently requested to debug a pre-.NET application Visual Basic, so I had to set up the environment for Windows 98 development… but nowdays all that software is deprecated and has no support. This is what I learned from my experience to get it up and running, but in case you can make a choice, I’d recommend you to rewrite the code in .NET, as this kind of DLL code relies platform, and things have changed a lot in 20 years.

1.- Delete all the files from the previously failed VB6 installation attempts

Be careful to avoid deleting the recent Visual Studio versions, as they may have similar paths.

The Visual Studio 6.0 files are by default under ‘C:\Program Files (x86)’ in 64 bits systems.

C:\Program Files (x86)\Microsoft Visual Studio\Common
C:\Program Files (x86)\Microsoft Visual Studio\MSDN
C:\Program Files (x86)\Microsoft Visual Studio\MSDN98
C:\Program Files (x86)\Microsoft Visual Studio\VB98
C:\Program Files (x86)\Microsoft Visual Studio\VC98
C:\Program Files (x86)\Microsoft Visual Studio\*.HTM
C:\Program Files (x86)\Microsoft Visual Studio\*.TXT
C:\Program Files (x86)\Common Files\Microsoft Shared\MSDesigners98
C:\Program Files (x86)\Common Files\Microsoft Shared\MSDN
C:\Program Files (x86)\Common Files\Microsoft Shared\VS98
C:\Program Files (x86)\Common Files\Microsoft Shared\Wizards98

Clean up the Windows registry entrys: run ‘regedit.exe’ and delete the following keys if they exist.

HKEY_LOCAL_MACHINE\Software\Microsoft\DevStudio
HKEY_LOCAL_MACHINE\Software\Microsoft\HTML Help Collections
HKEY_LOCAL_MACHINE\Software\Microsoft\MSVSDG
HKEY_LOCAL_MACHINE\Software\Microsoft\Visual Basic\6.0
HKEY_LOCAL_MACHINE\Software\Microsoft\Visual Component Manager
HKEY_LOCAL_MACHINE\Software\Microsoft\Visual Modeler
HKEY_LOCAL_MACHINE\Software\Microsoft\VisualStudio\6.0
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\DevStudio
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\HTML Help Collections
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\MSVSDG
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\Visual Basic\6.0
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\Visual Component Manager
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\Visual Modeler
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\VisualStudio\6.0
HKEY_CURRENT_USER\Software\Microsoft\DevStudio
HKEY_CURRENT_USER\Software\Microsoft\MSVSDG
HKEY_CURRENT_USER\Software\Microsoft\Visual Basic\6.0
HKEY_CURRENT_USER\Software\Microsoft\Visual Modeler
HKEY_CURRENT_USER\Software\Microsoft\VisualFoxPro
HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\6.0

2.- Modify the installation files to adapt them to current tech

First of all you will need a copy of the Visual Studio installer in your hard drive. I copied the content of an old college licensed CD into a folder, and proceeded to edit.

  • Open ‘SETUPWIZ.INI’ with a text editor (e.g. Notepad++), and replace ‘VmPath=ie4\msjavx86.exe’, which tries to install a really old Java implementation and makes the installation process fail, with an empty va. So the first part of the file should look like this:
[setup wizard]
eula = eula.txt
NTSP = NTsp3\nt4sp3_i.exe
NTSpMinVer = 3
IE4 = ie4\ie4setup.exe
CommonFilesMin = 50
IEIni=ie4check.ini
WFCClean = setup\wfcclean.exe
readme = readmevs.htm
pid = setup.ini
MSDN = setup.exe
Acme = acmboot.exe
AcmeId = vs98ecd1.inf
STF = setup\vs98ent.stf
DCOM98 = dcom98\dcom98.exe
MSDNID = msdn3?1.inf
NtSpUrl = ftp://ftp.microsoft.com/bussys/winnt/winnt-public/fixes/
IeUrl = http://www.microsoft.com/ie/ie40/download/
UsrUrl = http://msdn.microsoft.com/vstudio/register/default.htm
RegUrl = http://www.microsoft.com/isapi/redir.dll?Prd=vstudio&amp;Pver=98&amp;Ar=register
VmPath=
  • You must also edit the ‘SETUP.EXE’ properties. Go to the context menu of the file (right click as default), select properties, and got to the ‘compatibility’ tab. Check that you have selected:
    • Compatibility mode: execute as ‘Windows XP (Service Pack 3)’.
    • Configuration: ‘execute this program as administrator’.

3.- Execute the wizard installer

  1. Open the context menu of ‘SETUP.EXE’ and choose ‘run as administrator’.
  2. Don’t install ‘Source Safe’, as it fails.
  3. When we get to ‘choose the installation mode’ select ‘Custom’. Then follow these steps:
  • Do not install (as they fail):
    • Microsoft Visual FoxPro 6.0
    • Microsoft Visual InterDev 6.0
    • Microsoft Visual SourceDafe 6.0
    • ActiveX (obsolete version, generates conflict with the current version)
  • Install the unicode libraries: from the custom main menu, select the text ‘Microsoft Visual C++ 6.0’, and the button ‘Change option’ on the right side will be set as active. Click on it and follow a similar process for ‘VC++ MFC and Template Libraries’ and ‘MS Foundation Class Libraries’. Finally select all these options:
    • Static libraries
    • Shared libraries
    • Static libraries for Unicode
    • Shared libraries for Unicode
    • Browser database
    • Source code
  • Install the database: from the main ‘Custom’ menu, click on the ‘Data Access’ text , and the button ‘Change option’ on the right side will be set as active. Click on it and make sure that ‘ADO, RDS and OLE DB Providers’ is not selected. You will get a warning message saying that this component is esential for the application, but you should ignore it, as it will crash on Windows 10. Do select only the following options:
    • Microsoft ODBC Drivers
    • Remote Data Objects and Controls
    • Data environment
  • Install the tools: from the main ‘Custom’ menu, click on the ‘Enterprise Tools’ text, and the button ‘Change option’ on the right side will be set as active. Check that ‘Visual Studio Analyzer’ is not selected. Therefore, select only:
    • Aplication Performance Explorer
    • Repository
    • Visual Component Manager
    • Visual Basic Enterprise Components
    • VC++ Enterprise Tools
    • Microsoft Visual Modeler
  • As the last step, before pressing on ‘Finish’, do not let the program configure the environment vars.
    • If you have waited more than 5 minutes and the program is still ‘configuring the system’, you can assume something has gone wrong and the install has been frozen somewhere. Cancel it, clean up (see the first section on top of this post) and start all over again, reading carefully the steps.
    • If there is an error message about the Java machine you can ignore it. At that point you should be able by then to run ‘Visual Basic’ without the MSDN help package, so you get the bare bones yet fully functional experience.

4.- Execute the application

Run it always in administrator mode (right click on Visual Basic 6.0, and select ‘run as administrator’). Then, here it is, ready to run and debug ancient DLLs.

Visual Studio 6 on windows 10
After so many trial and errors, here it is: Visual Basic 6.0

 

❗️ If you want it the program to run using a Microsoft Office 2010 instance, you will need to load some dependencies. Go to ‘Project/References’ and select:

  • Visual Basic for Applications
  • Visual Basic runtime objects and procedures
  • Visual Basic objects and procedures
  • OLE Automation
  • Microsoft Excel 15.0 Object Library
  • Microsoft Word 15.0 Object Library

5.- References

My sources should get the credit they deserve for their help:

Browser integration in Java GUIs

Working around the Adobe PDF in 64 bits

Publicado en CodingMarkers el 12 de noviembre del 2017

Browser integration in Java GUIs

One of the challenges I had to face a couple of years ago was to migrate a Java desktop application from 32 bits to 64 bits architecture, and what may have been as simple as using the right JDK, switching to the proper external libraries versions and getting rid of the deprecated code actually had an extra issue. The application embeded a 32 bits PDF reader on a panel, and that limitation had been stopping this migration for a while.

One of the challenges I had to face a couple of years ago was to migrate a Java desktop application from 32 bits to 64 bits architecture, and what may have been as simple as using the right JDK, switching to the proper external libraries versions and getting rid of the deprecated code actually had an extra issue. The application embeded a 32 bits PDF reader on a panel, and that limitation had been stopping this migration for a while.

Going into the code itself, it was using the XULRunner, one of the Firefox engine components, so what it actually did was “using a 32 bits Firefox browser embedded”. It also used Adobe Acrobat generated interactive forms, which actually required Adobe Acrobat Reader itself to be rendered properly and this took Apache PDFBox out of the picture to do this task.

When I looked into the philosophy of running an embed browser I tried to take what happened in my laptop: when I was surfing the net and opening a PDF file my Firefox 32 bits browser, I didn’t use some browser code: I delegated this in the Adobe Acrobat Reader X Plugin for the browser. So, why should not we take this idea one step further, and use the same plugin through an integration? Adobe itself provides this system, so there would be no incompatibility issue, I just had to setup this through the operating system configuration to get a workaround and get the proper version for the machine architecture in which we are currently running the program.

So let’s meet DJNativeSwing, a library based in SWT (Standard Widget Toolkit) which allows us to have our very own embedded browser in our code. SWT is highly recommended library due to its portability, as it accesses to the Native Operating System GUIs, and that is exactly what we require for this kind of issue. It is also the next wrapping level to Java Swing, and actually lighter and faster, and I had already used it back in the day to avoid problems when I had to deal with Macromedia Flash plugin integrations.

As an example of how to configure this, we are going to follow 2 basic steps:

  • Set up a browser tab on a Java swing component.
  • Be able to open a PDF on that said browser tab.

Previous set up: the Maven dependecies required

Let’s start with the basics: first of all we will get the maven dependencies.

❕This was the stable version when the post was originally written.

<dependecy>
  <group>chrriis.dj.nativeswing</group>
  <artifactId>DJNativeSwing</artifactId>
  <version>1.0.2</version>
</dependecy>

 

Building a basic Swing interface

Right after that, we will define the basic window frame, with a button the select the file (via FileChooser) and a panel to show the results.

❕Setting up all the text strings as constants makes them easier to spot them in order to replace them if needed. It’s not really necessary, but doing it improves reusability.

import java.awt.BorderLayout;
import java.awt.Component;
import java.awt.FlowLayout;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.io.IOException;
import java.util.Map;

import javax.swing.BoxLayout;
import javax.swing.JButton;
import javax.swing.JFileChooser;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.JScrollPane;
import javax.swing.JTextPane;
import javax.swing.SwingUtilities;
import javax.swing.filechooser.FileNameExtensionFilter;

import nativeSwing.BrowserPanel;
import pdfHandler.PdfReader;
import xmlHandler.XMLHandler;
import chrriis.common.UIUtils;
import chrriis.dj.nativeswing.swtimpl.NativeInterface;

public class DemoPDFRenderLauncher {

  private static final String TITLE = "PDF Renderer demo";
  private static final String NO_OUTPUT_MESSAGE = "No output available";
  private static final String NO_DATA_MESSAGE = "There is no data available from form";
  private static final int LENGTH = 800;
  private static final int WIDTH = 600;
  private static final String FILTER_FILES "PDF files";
  private static final String FILE_EXTENSION "pdf";

  /**
   * The main app window
   */
  private JFrame window;
  /**
   * The path of the file we will open
   */
  private String path;
  /**
   * Button for open file function
   */
  private JButton buttonOpen;
  /**
   * A browser panel
   */
  private BrowserPanel browserPanel;

  /**
   * Constructor method, creates the GUI
   */
  public Launcher() {
    window = new JFrame(TITLE);
    window.getContentPane().setLayout(new BorderLayout());
    window.setSize(LENGTH, WIDTH);
    window.add(createButtonsPanel(), BorderLayout.NORTH);
    window.add(createContentPanel(), BorderLayout.CENTER);
    window.setVisible(true);
    window.addWindowListener(new WindowAdapter() {
      public void windowClosing(WindowEvent e) {
NativeInterface.close();
        System.exit(0);
      }
    });
  }

  /**
   * Creates a button panel with the action button: open a file
   * @return the buttons panel
   */
  private Component createButtonsPanel() {
    JPanel panel = new JPanel();
    panel.setLayout(new FlowLayout());
    buttonOpen = new JButton("Open file");
    buttonOpen.addActionListener(new ButtonOpenController());
    panel.add(buttonOpen);
    return panel;
  }

  /**
   * Creates a panel to render the content
   *
   * @return the buttons panel
   */
  private Component createContentPanel() {
    JPanel panel = new JPanel();
    panel.setLayout(new BoxLayout(panel, BoxLayout.Y_AXIS));
    JScrollPane scrollPaneText = new JScrollPane(textPanel);
    panel.add(scrollPaneText);
    //here we will insert the DJNativeSwing panle
    browserPanel = new BrowserPanel();
    JScrollPane scrollPaneBrowser = new JScrollPane(browserPanel);
    panel.add(scrollPaneBrowser);
    return panel;
  }

  /**
   * Load button controller, which launches the FileChooser.
   */
  private class ButtonOpenController implements ActionListener {
    @Override
    public void actionPerformed(ActionEvent arg0) {
      launchOpenSelectFile();
    }
  }

  /**
   * Launches the FileChooser window, and invokes the pdf opener window.
   */
  private void launchOpenSelectFile() {
    JFileChooser fileChooser = new JFileChooser();
    fileChooser.setAcceptAllFileFilterUsed(false);
    FileNameExtensionFilter filter = new FileNameExtensionFilter(
FILTER_FILES, FILE_EXTENSION);
    fileChooser.addChoosableFileFilter(filter);
    if (fileChooser.showOpenDialog(window) == JFileChooser.APPROVE_OPTION) {
      path = fileChooser.getSelectedFile().getAbsolutePath();
browserPanel.navigate(path);
    }
  }

  @SuppressWarnings("unused")
  public static void main(String[] args) {
    UIUtils.setPreferredLookAndFeel();
    NativeInterface.open();
    SwingUtilities.invokeLater(new Runnable() {
      public void run() {
        Launcher demo = new Launcher();
      }
    });
  }
}

Embeding a web browser

Then let’s follow up with browser tab: if we just wanted to create a browser panel it would be as easy as writing the following lines:

import java.awt.BorderLayout;

import javax.swing.BorderFactory;
import javax.swing.JPanel;

import chrriis.dj.nativeswing.swtimpl.components.JWebBrowser;

public class BasicBrowserPanel extends JPanel {

  private static final String TITLE = "";

  /**
   * The browser will be handled in this specific component
   */
  private JWebBrowser webBrowser;

  /**
   * Constructor
   */
  public BrowserPanel() {
    super(new BorderLayout());
    JPanel webBrowserPanel = new JPanel(new BorderLayout());
webBrowserPanel.setBorder(BorderFactory.createTitledBorder(TITLE));
    webBrowser = new JWebBrowser();
    webBrowser.setBarsVisible(false);
    webBrowser.setStatusBarVisible(false);
    webBrowserPanel.add(webBrowser, BorderLayout.CENTER);
    add(webBrowserPanel, BorderLayout.CENTER);
  }

  /**
   * Initializes the browser and sets a value in the URL storage
   * @param path the URL value o file path to open
   */
  public void navigate(String path) {
    webBrowser.setVisible(true);
    webBrowser.navigate(path);
  }

  /**
   * Makes the browser retrieve and render the content from the path previously stored
   */
  public String getAddress(){
    return webBrowser.getHTMLContent();
  }

  /**
   * Hides the browser controls (forward, back, home buttons...)
   */
  public void hideContent() {
    webBrowser.setVisible(false);
  }
}

This way we are getting a fully functional web browser, very similar to the browser in the Eclipse IDE, but with too many unnecessary functions for what we are trying to do here. Since we are only doing the rendering process by delegating it on Adobe, we can remove the GUI extra elements from this whole custom browsing system and leave the bare panel.

import java.awt.BorderLayout;
import java.awt.Component;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;

import javax.swing.JFrame;
import javax.swing.JOptionPane;
import javax.swing.JPanel;
import javax.swing.SwingUtilities;

import chrriis.dj.nativeswing.swtimpl.NativeInterface;
import chrriis.dj.nativeswing.swtimpl.components.JWebBrowser;

/**
 * Allows to launch a JFrame containing an embedded browser.
 */
public class BrowserFrameLauncher {

  /**
   * Renders the file content on a browser via DJ Native Swing
   *
   * @param path
   * the file url if we pass as a parameter any webpage URL, the system would try to render it
   * @return a JPanel with a native browser, to render the file content
   */
  private Component createBrowserPanel(String path) {
    JWebBrowser.useXULRunnerRuntime();
    JPanel fileBrowserPanel = new JPanel(new BorderLayout());
    final JWebBrowser fileBrowser = new JWebBrowser();
    fileBrowser.setBarsVisible(false);
    fileBrowser.setStatusBarVisible(false);
    fileBrowser.navigate(path);
    fileBrowserPanel.add(fileBrowser, BorderLayout.CENTER);
    return fileBrowserPanel;
  }
}

Setting the last piece of the puzzle: getting the PDF itself

Finally let’s make the final touches to open a PDF. What we are actually doing is getting the PDF file path into the browser, so in the end we have a new layer over our old friend XULRunner, but this provides us a way to integrate the plugins via the “right architecture version” SWT library. So as a conclussion we are able to connect to the “right architecture version” plugin, fixing our rendering problem and making us independent from the 32 bits plaform once and for all.

// excerpt from BrowserFrameLauncherPDF.java

  private static final String CLOSING_MESSAGE = "Do you really want to close the file?";
  private static final String RENDERED_TITLE = "PDF Renderer demo - Embed Browser";

  /**
   * Opens a file and shows it content in a JFrame.
   *
   * @param path
   * the url of the file to open in a JFrame
   */
  public static void openPDF(final String path) {
    NativeInterface.open();
    SwingUtilities.invokeLater(new Runnable() {
      public void run() {
        final JFrame frame = new JFrame(RENDERED_TITLE);
frame.setLocation(0, 0);
        //we may set up a default size for this test
        //frame.setSize(800, 600);
        frame.setVisible(true);
        frame.add(createBrowserPanel(path));

        //a window listener would allow us to control the closing actions
        frame.addWindowListener(new WindowAdapter() {
          public void windowClosing(WindowEvent e) {
            int i = JOptionPane.showConfirmDialog(frame,CLOSING_MESSAGE);
            if (i == 0) {
              NativeInterface.close();
            }
          }
        });
      }
    });
  }

❗️ Please notice the NativeInterface.open() line to make sure about getting the components correctly booted, and the threading of this component in order to avoid other processes interfering with the rendering.

Spider-files

Or how to extract the PDF content from folders in a recursive way

Publicado en CodingMarkers el 1 de septiembre del 2017

Playing with the files on a Spider-man way

When we are talking about processing information, it all starts with getting the input data. This matter usually comes down to explore some directory (or repository) containing different sets of information, and then “do something” with it. Those repositories usually tend to be massive, so the most effective thing to do is automating the process. We don’t need superpowers to achieve this, only a few pointers are enough to make this goal reachable.

As an example for this process, I’ll explain how to recover all the content from a tree of folders containing PDF files in a very simple way. In order to sort this out, I’ll divide the problem in three main issues:

  • Getting the file content of a single PDF file.
  • Getting all the folders and sub-folders paths.
  • Create a loop so we retrieve all the PDF text from all the files.

1. Processing the PDFs

One of the usual file formats we get is the Adobe Acrobat PDF (Portable Document Format). This format was created with the intent of being independent from application software, hardware and operating system, by storing not only the text and graphics, but the whole information about the layout and fonts. There are multiple readers, such as the Adobe Acrobat Reader, Evince or Foxit Reader.

Alternative options to read PDF files
Alternative options to read PDF files

 

Of course, not everything is so pretty, as the some Adobe PDF files contains XFDF (XML Forms Data Format) and therefore they are only properly rendered in proprietary Adobe programs for now. I have faith in the open source community to eventually solve this issue, as this defeats the purpose of the format.

I would also like to point out that, while PDFs may be a standard for documents which will be printed out, they are not “screen-friendly”, meaning they cannot be adapted properly to be read on e-books, tablets and smartphones in a comfortable way, as they are not able to adjust the content size to the screen. My advice is that if you are publishing a document, you may want to consider the EPUB format for the digital edition.

Every single document processing application starts with getting some source data, and in this case, we are going to use the Apache PDFBox package, an open source library which provides us several different functions such as:

  • Create new PDF documents.
  • Extract the contents from existing documents.
  • Manipulate a given document.
  • Digitally sign PDF files.
  • Print a PDF file using the standard Java printing API.
  • Save PDFs as image files, such as PNG or JPEG.
  • Validate PDF files against the PDF/A-1b standard.
  • Encrypt/decrypt a PDF document.

In this example I am only going to work with plain text, as this is an excerpt from a program where I intended to make the text indexable in order to create search relationships between different documents, so bear in mind that PDFBox can do so much more than that.

So let’s get down to business: the very first step if we are using Maven is adding the required dependencies to the pom.xml file, so we can get the library.

❕This was the stable version when the post was originally written.

<dependency>
  <groupId>org.apache.pdfbox</groupId>
  <artifactId>pdfbox</artifactId>
  <version>2.0.6</version>
</dependency>

Now we can work on a very short and simple snippet to read the text content from a PDF file, and store it on a String so we are able to do some the heavy processing work afterwards such as using Lucene later to index that said content and create some search functions to improve the access to information.

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;

import org.apache.pdfbox.cos.COSDocument;
import org.apache.pdfbox.pdfparser.PDFParser;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.util.PDFTextStripper;

public class PdfReader {

  public static String extractTextFromPdf(String path) throws IOException {
    System.out.println("Parsing a PDF");
    String parsedText = "";

    File f = new File(path);
    if (!f.isFile()) {
      System.err.println("The file " + path + " doesn't exist.");
      return null;
    }

    PDFParser parser = new PDFParser(new FileInputStream(f));
    parser.parse();
    COSDocument cosDoc = parser.getDocument();
    PDFTextStripper pdfStripper = new PDFTextStripper();
    PDDocument pdDoc = new PDDocument(cosDoc);
    parsedText = pdfStripper.getText(pdDoc);

    return parsedText;
  }

}

2. The Spider and its web

When we go through a folder or directory, we may find not only files, but other sub-folders, which may also contain more files or more subdirectories, and so on. The consequence of this is that we would need a way to go through all this hierarchical structure, using a recursive function. This idea would be the core of the “Spider” which will go through the “web” of files:

+ Directory_1
|-- File_1.pdf
|-- File_2.pdf
|-+ Directory_2
  |-- File_3.pdf
  |-- File_4.pdf

The “Spider” will detect all the files (File_1.pdf, File_2.pdf, File_3.pdf and File_4.pdf) thanks to a recursive structure, instead of getting stuck with only the first level of the tree (File_1.pdf and File_2.pdf).

This can be summarized in the following algorithm structure:

  1. Initialize the while loop for each element
  2. Is it a base case?
    • Yes: solve base case
    • No: execute recursive function
  3. End while loop

We can achieve this in Java by relying only on the java.io and java.util libraries, which are included in every Java Development Kit (JDK).

import java.io.File;
import java.util.LinkedList;
import java.util.List;

public class Spider {

  /*
   * Lists the files only on that level
   */
  public List<String> listPaths(String path) {
    File f = new File(path);

    List<String> l = new LinkedList<String>();
    if (f.exists()) {
      File[] fileArray = f.listFiles();
      for (int i = 0; i<fileArray.length; i++) {
        l.add(fileArray[i].getAbsolutePath());
      }
    } else {
      System.err.println("The path " + path + " is incorrect");
    }
    return l;
  }

  /*
   * Also lists the sub-directories content
   */
  public List<String>; listPathsRecursive(String path) {
    File f = new File(path);
    List<String> l = new LinkedList<String>();
    if (f.exists()) {
      File[] fileArray = f.listFiles();
      for (int i = 0; i < fileArray.length; i++) {
        // check the sub-directories
        if (fileArray[i].isDirectory()) {
          List<String> l1 = listPathsRecursive(fileArray[i].getAbsolutePath());
          l.addAll(l1);
        } else {
          //isValidFormat will check the file extensions
          //e.g. fileNameString.endsWith(".pdf")
          if (ClasificadorDeFicheros.isValidFormat(ficheros[i]
.getAbsolutePath())) {
            l.add(ficheros[i].getAbsolutePath());
          }
        }
      }
    } else {
      System.err.println("The path " + path + " is incorrect");
    }
    return l;
  }
}

3. Loop using the methods from steps 1 and 2

Finally we get to the easiest part: we just need some basic Java iterations to finish our epic SpiderPdf.java: we get all the files paths with the method from the second step, and process it by invoking the code generated on the first step.

// excerpt from MapFolderOfPdfs.java
Iterator<String> it = spider.listPaths(mainFolderPath);
Map<String, String> mapContent = new HashMap<String, String>();
while(it.asNext){
  String currentPath = it.next();
  mapContent.put(currentPath, PdfReader.extractTextFromPdf(currentPath));
}