Installing and configuring vsftpd

To perform ftp file transaction it is required to install a ftp server and configure it appropriately. The vsftpd server is used in this scenario.

First install the vsftpd server – sudo apt-get install vsftpd

Configure the vsftpd server by editing the /etc/vsftpd.conf file

listen=YES

anonymous_enable=NO

local_enable=YES

write_enable=YES

anon_mkdir_write_enable=NO

dirmessage_enable=YES

use_localtime=YES

xferlog_enable=YES

secure_chroot_dir=/var/run/vsftpd/empty

pam_service_name=vsftpd

rsa_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem

rsa_private_key_file=/etc/ssl/private/ssl-cert-snakeoil.key

pasv_enable=YES

pasv_min_port=11810

pasv_max_port=11819

pasv_address=<public external ip address of the machined where vsftpd is installed>

 

Open the port ranges 20-21 and 11810-11819 of the machine

Finally restart the vsftp server – sudo service vsftpd restart.

 

FTP user accounts

I created  two user accounts to send and receive files with login disabled (This is not required)

sudo adduser ftpuser1 –disabled-login

sudo adduser ftpuser2 –disabled-login

 

Create the following directory structure

/home/ftpuser1/files/in      /home/ftpuser1/files/out

/home/ftpuser2/files/in      /home/ftpuser2/files/out

 

Change the change file owner to respective users

chown -R ftpuser1 /home/ftpuser1/files

chown -R ftpuser2 /home/ftpuser2/files

Configure Apache2 and NginX as Content based router

Content based routing is to send received request to particular destination depending on the request content. The configuration bellow will be using a custom HTTP header as the content to decide the destination

Apache2

Create a file named a2cbr in /etc/apache2/sites-available, and copy the following content to a file named a2cbr

<VirtualHost *:10000>
    ServerName localhost
    ProxyRequests Off
    RewriteEngine On

    <Directory>
        AllowOverride All
        <Limit GET HEAD POST PUT DELETE OPTIONS>
        Order Allow,Deny
        Allow from all
        </Limit>
    </Directory>

    RewriteCond %{HTTP:routing} xadmin
    RewriteRule ^/a2cbr(.*) http://localhost:9000/service/EchoService [P]
</VirtualHost>

NginX

Create a file named nginxcbr in /etc/nginx/sites-available, and copy the following content to a file named nginxcbr

server {
   listen *:15000;
   server_name localhost;

   location  / {
       if ($http_routing ~ 'xadmin') {
          rewrite ^/nginxcbr(.*) /service/EchoService;
           proxy_pass http://localhost:9000;
       }
    }
}
<pre>

JMX tips

I’m hoping to continuously update this post as I come across important facts while developing JMX applications. So here we go…..

  1.  Do not pass complex structures to the JMX connection. JMX finds it difficult to render complex objects.
    e.g: Last week I was trying to pass a hash map which stored several other hash maps as its values, using MXBeans (The purpose was to add custom statistic support to UlraESB). 

    HashMap<String, HashMap<String, Integer>> statisticMap  = new HashMap<String, HashMap<String, Integer>>();
    

    The  statisticMap  was directly passed to the JMX connection. However the beans were created and visible through  the jconsole, but data wasn’t rendered. Then I tried out the following

    List<HashMap<String, Integer>> statisticList  = new ArrayList<HashMap<String, Integer>>();
    

    The list was directly passed to the connection and it worked like magic. Data were properly rendered. The only difference between these is that first design is “Maps in a Map” and the second is “Maps in a List”. I cannot explain why this happens… Please share your thoughts on this

  2. The objects which are passes to the JMX connection should be Serializable. If a static object is passed then the JMX connection would not be able to expose the data of that static object, since  static objects are not serializable. (Serialization can be only applied to objects and static variables does not belong to individual instances, they are class variables)

How TO KILL an ANT

Well this is nothing related my tech attempts, but I found this story really funny and thought of re-posting it on my blog . Also reminded me of my school days, where we  invented new formulas and theories just to get 15 marks 😀  The moral of this story is nothing but the absolute truth.

So here we go “How to kILL an ant” 😉
Q: How to Kill an Ant?
Asked in exam for 15 marks..
Student’s Answer:
Mix Chilli Powder with Sugar & keep it outside the Ant’s Hole.
After eating, Ant will search for some water near a water tank.
Push ant in to it. Now ant will go to dry itself near fire.
When it reaches fire, put a bomb into the fire.
Then admit wounded ant in ICU.
Remove oxygen mask from its mouth and kill the ant.

MORAL: Don’t play with students they can do anything for 15 marks.

Writing sample configurations for Apache ServiceMix

ServiceMix is an open source ESB by Apache Software Foundtion. This post will provide a brief description on creating new sample configurations and running them with ServiceMix

I have created sample configurations for five different scenarios.

The Direct Proxy

This is a basic scenario where a client issues a POST request and the ESB sends request to a specified service without any changes to the original message.
This cofiguration needs a http-consumer and a http-provider.

Steps to create the project

  • Create a folder and name it “direct-proxy”
  • Add a pom.xml file to the folder with following configuration
<?xml version="1.0" encoding="utf-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <groupId>org.apache.servicemix.http.proxy</groupId>
        <artifactId>parent</artifactId>
        <version>1.0-SNAPSHOT</version>
        <packaging>pom</packaging>
        <name>SMX-HTTP :: DirectProxy</name>
        <url>http://servicemix.apache.org</url>
     </project>
  • cd direct-proxy
  • Create a comsumer service unit
mvn archetype:create -DarchetypeArtifactId=servicemix-http-consumer-service-unit -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=proxy-consumer-su
  • Create a provider service unit
mvn archetype:create -DarchetypeArtifactId=servicemix-http-provider-service-unit -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=proxy-provider-su
  • Create the Service Assembly
mvn archetype:create -DarchetypeArtifactId=servicemix-service-assembly -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=proxy-sa

After executing the above commands you will find three folders created in the ‘direct-proxy’ folder.

  • Navigate to the ‘proxy-sa’ folder and add the created service units as dependencies to the pom.xml file.

e.g ;

<dependencies>
    <dependency>
         <groupId>org.apache.servicemix.http.proxy</groupId>
         <artifactId>proxy-provider-su</artifactId>
         <version>1.0-SNAPSHOT</version>
    </dependency>
    <dependency>
        <groupId>org.apache.servicemix.http.proxy</groupId>
        <artifactId>proxy-consumer-su</artifactId>
        <version>1.0-SNAPSHOT</version>
    </dependency>
</dependencies>
  • Then navigate to ‘proxy-consumer-su’ and ‘proxy-provider-su’ and check if the generated pom.xml files have specified the componentName. If not specify the component name as follows.
<properties>
    <componentName>servicemix-http</componentName>
</properties>

Now you are ready to deploy the configuration.

  • Do the necessary changes to the xbean files in ‘proxy-consumer-su’ and ‘proxy-provider-su’.

{source}

  • Start servicemix –
    1. cd <servicemix-home>/bin
    2. ./servicemix
  • Build the project
    1. cd direct-proxy
    2. mvn clean install

The project should build without any errors

  • Navigate to direct-proxy/proxy-sa/target. Copy the zip file and paste it to <servicemix-home>/deploy folder.

The Content Based Router

The content based router will evaluate the POST request issued by the client. If the evaluation passes the request is sent to a specified service.
This configuration evaluates the request using xpath. Apache Camel is used to define routes and to perform the xpath evaluation.

Steps to create the project

  •  Create a folder and name it as “router”
  • Add a pom.xml file to the folder with following configuration
<?xml version="1.0" encoding="utf-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>org.apache.servicemix.cbr.camel</groupId>
    <artifactId>parent</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>pom</packaging>
    <name>SMX-CAMEL :: CBRProxy</name>
    <url>http://servicemix.apache.org</url>
</project>
  •  cd router
  • create the ServiceMix Camel service unit
mvn archetype:create -DarchetypeArtifactId=servicemix-camel-service-unit -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=cbr-camel-su
  • Create a comsumer service unit
mvn archetype:create -DarchetypeArtifactId=servicemix-http-consumer-service-unit -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=cbr-consumer-su
  • Create a provider service unit
mvn archetype:create -DarchetypeArtifactId=servicemix-http-provider-service-unit -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=cbr-provider-su
  • Create the Service assembly
mvn archetype:create -DarchetypeArtifactId=servicemix-service-assembly -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=cbr-sa

There will be four folders created inside the ‘router’ folder

  •  Navigate to the ‘cbr-sa’ folder and add the created service units as dependencies to the pom.xml file.(refer the direct proxy)
  • Then navigate to ‘cbr-consumer-su’ and ‘cbr-provider-su’ and check if the generated pom.xml files have specified the componentName. If not, specify the  component name as follows.
<properties>
    <componentName>servicemix-http</componentName>
</properties>

Also check the pom generated in ‘cbr-camel-su’. If the componenName is not specified, if not specify it as follows

<properties>
    <componentName>servicemix-camel</componentName>
</properties>

To get this configuration working I had to replace the following of the pom.xml in cbr-camel-su

<dependency>
    <groupId>org.apache.servicemix</groupId>
    <artifactId>servicemix-camel</artifactId>
    <scope>provided</scope>
</dependency>

with

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-core</artifactId>
    <version>2.8.0</version>
</dependency>

Before this replacement I could not deploy the project due to “Error creating bean with name ‘camel’: Invocation of init method failed; nested exception is java.lang.NoSuchMethodError: org.apache.servicemix.cbr.camel.MyRouteBuilder.from”

Now you are ready to deploy the configuration.

  • Do the necessary changes to the xbean files, camel contex and MyRouteBuilder.

{source}

  • Start servicemix – cd <servicemix-home>/bin

./servicemix

  • Build the project

cd router
mvn clean install
The project should build without any errors

  • Navigate to direct-proxy/proxy-sa/target. Copy the zip file and paste it to <servicemix-home>/deploy folder.

XSLT Transformation

This configuration transforms POST request using a XSLT stylesheet in the ESB, then sends the transformed request to an echo service. The echo service echo backs the received request. The echoed back transformed request is again transformed back to the original state, using another XSLT style sheet and sent back to the client. The Camel pipeline pattern is used for this.

  • Create a folder and name it as “xslt”
  •  add a pom.xml file to the folder with following configuration
 <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.adroitlogic.com</groupId>
<artifactId>parent</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>
<name>SMX-CAMEL :: XSLT</name>
<url>http://servicemix.apache.org</url>
</project>

  • cd xslt
  • create the camel service unit
mvn archetype:create -DarchetypeArtifactId=servicemix-camel-service-unit -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=transform-camel-su
  • create transform-back-xslt-su
mvn archetype:create -DarchetypeArtifactId=servicemix-saxon-xslt-service-unit -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=transform-back-xslt-su
  • creat transform-xslt-su
mvn archetype:create -DarchetypeArtifactId=servicemix-saxon-xslt-service-unit -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=transform-xslt-su
  • create transform-xslt-comsumer-su
mvn archetype:create -DarchetypeArtifactId=servicemix-http-consumer-service-unit -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=transform-xslt-consumer-su
  • create transform-xslt-provider-su
mvn archetype:create -DarchetypeArtifactId=servicemix-http-provider-service-unit -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=transform-xslt-provider-su
  • create the SA
mvn archetype:create -DarchetypeArtifactId=servicemix-service-assembly -DarchetypeGroupId=org.apache.servicemix.tooling -DartifactId=transform-sa

There will be four folders vreated inside the ‘router’ folder

  • Navigate to the ‘cbr-sa’ folder and add the created service units as dependencies to the pom.xml file.

        org.apache.servicemix.xslt.transform
        transform-back-xslt-su
        1.0-SNAPSHOT

        org.apache.servicemix.xslt.transform
        transform-camel-su
        1.0-SNAPSHOT

        org.apache.servicemix.xslt.transform
        transform-xslt-consumer-su
        1.0-SNAPSHOT

        org.apache.servicemix.xslt.transform
        transform-xslt-provider-su
        1.0-SNAPSHOT

        org.apache.servicemix.xslt.transform
        transform-xslt-su
        1.0-SNAPSHOT

  •  Then navigate to ‘transform-xslt-consumer-su’ and ‘transform-xslt-provider-su’ and check if the generated pom.xml files have specified the componentName. If not, specify the component name as follows.
servicemix-http

Check the pom generated in ‘transform-camel-su’. If the componenName is not specified, if not specify it as follows


     servicemix-camel

Check the pom generated in ‘transform-back-xslt-su’ and ‘transform-xslt-su’. If the componenName is not specified, if not specify it as follows


     servicemix-saxon

  • Do the necessary changes to the xbean files, camel contex and MyRouteBuilder.
  • Start servicemix – cd <servicemix-home>/bin

./servicemix

  • Build the project

cd router
mvn clean install
The project should build without any errors

  •  Navigate to direct-proxy/proxy-sa/target. Copy the zip file and paste it to <servicemix-home>/deploy folder.

So this is how you create, build and deploy samples in ServiceMix. However I think the overhead is too much. It is needed to build a project each time a new sample is deployed, unlike in most other ESBs where deployin a new sample is just a matter of creating a sample configuration file and you are up and running 🙂

Capture Packets from Wireshark and see the content of requests or responses

First start Wireshark. If you are working on Ubuntu it is important to run Wireshark as the root. (I think same applies for other linux platforms as well)

sudo wireshark

Then capture options should be specified. Press the Capture Options button on the welcome page or the second button from the left in the tool bar.

Then specify the interface and the port number. I’m capturing packets within localhost, so the interface would be “lo” and the port which I’m receiving the request is 8192

Once you specify these options you are ready to capture packets. Just click on the start button.

When the request is sent, wireshark will display a list of captured packet.

Right click on one of the packets and select “Follow TCP Stream”.  The pop up window will show the request received and response sent

A Simple File Uploader Servlet

The HTML form

<form action="FileUpload?artifact=endpoints&type=items" enctype="multipart/form-data" method="post">
    <input type="file"/>
    <button type="submit">Upload</button>
</form>

 

The …./WEB-INF/web.xml

<web-app>
        .
        .
        .
  <servlet>
     <servlet-name>FileUpload</servlet-name>
     <servlet-class>org.adroitlogic.ultraesb.zabbix.FileUpload</servlet-class>
  </servlet>

  <servlet-mapping>
     <servlet-name>FileUpload</servlet-name>
     <url-pattern>FileUpload</url-pattern>
  </servlet-mapping>
        .
        .
        .
</web-app>

The form action value should be similar to the url-pattern of servlet-mapping in web.xml, i.e FileUpload. The rest of the string in form action specifies the url parameters. The question mark indicates the start of url parameters. Two url parameters are defined in this form action, artifact and type. The value of artifact is endpoints and the value of type is items.

The values of these url parameters can be retrieved in the servlet by request.getParameter(“urlParameter”) method. (servlet lines 24,25)

 

The Servlet

The third party jars used : Apache Commons File Upload and Apache Commons IO

import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.FileItemFactory;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.File;
import java.util.List;

public class FileUpload extends HttpServlet {

    private static final Logger logger = 
        LoggerFactory.getLogger(FileUpload.class);

    protected void doPost(HttpServletRequest request, HttpServletResponse 
        response) {

        StringBuilder basePath = new StringBuilder();
        basePath.append("conf").append(File.separator).append("zabbix").
            append(File.separator).append(request.getParameter("artifact")).
            append(File.separator).append(request.getParameter("type")).
            append(File.separator);

        FileItemFactory fileItemFactory = new DiskFileItemFactory();
        ServletFileUpload uploadHandler = new 
            ServletFileUpload(fileItemFactory);
        
        String destination = "the destination url that you want to send the 
            redirect to";
        response.setContentType("text/plain");

        try {
            List<FileItem> items = (List<FileItem>) 
                uploadHandler.parseRequest(request);
            for (FileItem item : items) {

                if (item.isFormField()) {

                    String fileName = item.getString();

                    int separatorIndex = fileName.lastIndexOf(File.separator);
                    if (separatorIndex > 0) {
                        fileName = fileName.substring(separatorIndex + 1);
                    }
                    basePath.append(fileName);
                    File file = new File( basePath.toString());
                    item.write(file);
                }                                       
           }
            response.sendRedirect(response.encodeURL(destination));
        } catch (Exception e) {
             logger.warn("Fault in uploading file", e);
        }
    }
}