Zabbix Monitoring via API


A few weeks ago I was asked to implement a feature for the Uconsole, to monitor “UltraESB” artifacts exposed to JMX connection. In today’s post I’ll be explaining the approach that I followed to achieve this task.

You can get an idea how the API works here.

The “item” is the basic element that should be configured to start Zabbix Monitoring. A graph is created with relevance to an item. Each item is uniquely identified by the name given and most importantly by its key. It describes the object to be monitored by the item.

This post will mainly focus on how to work with the zabbix api and how to interact with it. To get a further understanding, refer this link. It also describes how to specify the Zabbix item key for a JMX artifact.

Creating an item via the GUI is pretty easy, but the GUI is not very helpful always. Specially when several number of items and graphs need to be created at one shot. My requirement was the latter one.

Other third party JARs used

Apache Http Client
Jackson Java JSON-processor

The Design

If you went through the zabbix api it is made clear that first a user should get an authentication token to  register  items, graphs and triggers. So first we’ll get authorized.

public String authenticate(String username, String password, String url) {
    this.password = password;
    this.userName = username;
    zabbixApiUrl = url;

    StringBuilder uiConnectMessage = new StringBuilder();
    StringBuilder sb = new StringBuilder();
    sb.append("{\"jsonrpc\":\"2.0\"").
    append(",\"params\":{").
    append("\"user\":\"").append(username).
    append("\",\"password\":\"").append(password).
    append("\"},").
    append("\"method\":\"user.authenticate\",").
    append("\"id\":\"2\"}");

    try {
        HttpResponse response = postAndGet(sb.toString());
        HttpEntity entity = response.getEntity();

        HashMap untyped = mapper.readValue(EntityUtils.toString(entity), HashMap.class);
        auth = untyped.get("result");

        if (auth == null) {
            throw new IllegalArgumentException("Authorization failed to : " + url + ", using username : "               + username);
        }
        uiConnectMessage.append("Successfully connected to the server\n");

    } catch (IOException e) {
        uiConnectMessage.append("Could not connect to the Zabbix Server at : ").
        append(url).append(" Exception : ").append(e.getMessage()).append("\n");
    }
    return uiConnectMessage.toString();
}
    private HttpResponse postAndGet(String request) throws IOException {
        HttpClient client = new DefaultHttpClient();
        HttpPost httpPost = new HttpPost(zabbixApiUrl);
        httpPost.setEntity(new StringEntity(request));
        httpPost.addHeader("Content-Type", "application/json-rpc");
        return client.execute(httpPost);
    }

The above method will authenticate you to the zabbix server when the zabbix api url, user name and the password is given.
The received authentication token is stored in the “auth” varible which is declared globally for later uses.  Bellow is a screenshot of the web management console of the UltraESB which invokes the above methods


If you are successfully authenticated to the server, a list of hosts and a list of applications defined for each hosts are  presented. It is important to choose a host and a relevant application to register items. Defining an application is optional, but if you don’t latest data for an item will not be shown on the Zabbix GUI.

These two methods are used to retrieve hosts and applications defined to each hosts. When a host is selected from the drop down menu, the application drop down will show only the applications defined for the selected host.

    private Map<String, String> hostMap = new HashMap<String, String>();
    private Map<String, Map<String, String>> appMap = new HashMap<String, Map<String, String>>();

    private void getHosts() throws IOException {
        StringBuilder sb = new StringBuilder();
        sb.append("{\"jsonrpc\":\"2.0\",");
        sb.append("\"method\":\"host.get\",");
        sb.append("\"params\":{");
        sb.append("\"output\":\"extend\"},");
        sb.append(" \"auth\":\"").append(auth).append("\",");
        sb.append("  \"id\":2}");

        HttpResponse response = postAndGet(sb.toString());
        HttpEntity entity = response.getEntity();

        JsonNode rootNode = mapper.readValue(EntityUtils.toString(entity), JsonNode.class);
        JsonNode resultNode = rootNode.path("result");
        Iterator hostList = resultNode.getElements();
        while (hostList.hasNext()) {
            JsonNode element = hostList.next();
            hostMap.put(element.findValue("hostid").toString().replaceAll("(\\[|\\]|\")", ""), element.findValue("host")
                .toString().replaceAll("(\\[|\\]|\")", ""));
        }
    }

    private void getApps() throws IOException {
        StringBuilder sb = new StringBuilder();
        sb.append("{\"jsonrpc\":\"2.0\",");
        sb.append("\"method\":\"application.get\",");
        sb.append("\"params\":{");
        sb.append("\"output\":\"extend\"},");
        sb.append(" \"auth\":\"").append(auth).append("\",");
        sb.append("  \"id\":2}");

        HttpResponse response = postAndGet(sb.toString());
        HttpEntity entity = response.getEntity();

        JsonNode rootNode = mapper.readValue(EntityUtils.toString(entity), JsonNode.class);
        JsonNode resultNode = rootNode.path("result");
        Iterator appList = resultNode.getElements();
        while (appList.hasNext()) {
            JsonNode element = appList.next();
            JsonNode hosts = element.path("hosts");
            Iterator hostList = hosts.getElements();
            while (hostList.hasNext()) {
                JsonNode hostElement = hostList.next();
                String hostID = hostElement.findValue("hostid").toString().replaceAll("(\\[|\\]|\")", "");
                if (hostMap.containsKey(hostID)) {
                    if (appMap.containsKey(hostID)) {
                        appMap.get(hostElement.findValue("hostid").toString().replaceAll("(\\[|\\]|\")", "")).put(element.
                            findValue("name").toString().replaceAll("(\\[|\\]|\")", ""), element.findValue("applicationid").
                            toString().replaceAll("(\\[|\\]|\")", ""));

                    } else {
                        Map apps = new HashMap();
                        apps.put(element.findValue("name").toString().replaceAll("(\\[|\\]|\")", ""), element.
                            findValue("applicationid").toString().replaceAll("(\\[|\\]|\")", ""));
                        appMap.put(hostID, apps);
                    }
                }
            }
        }
    }

Finally when the user chooses the host and the application, the available artifacts are displayed. The user have the will of choosing which artifact is to be monitored.

Several items, graphs and triggers can be created for one type of artifact. For an example, some of the attributes exposed by the JMX connection of an Endpoint will be active address count, failed sending messages, ready address count etc. This program facilitates registration of many attributes for particular endpoint (not only endpoints other artifacts as well).

So how is this done?
I have defined a directory structure as shown in the image bellow

If a user needs to monitor a particular attribute, all what he have to do is to drop in json template file to the relevant folder, and the program will create items or graphs or triggers by reading the template files.
Following is a sample json file stored in the items directory of endpoints

NOTE : If a user checks an artifact and clicks the register button, the program will create items, graphs and triggers for all the json templates saved in the relevant artifact directory. (the Uconsole also provides with the facility of disabling a particular template).

{
"jsonrpc":"2.0",
"method":"item.create",
"params":{
      "description": "$filename$-$id$",
      "key_": "jmx[org.adroitlogic.ultraesb.detail:Type=Endpoints,Name=$id$][Details.activeAddressCount]",
      "hostid": "$hostid$",
      "value_type": 3 ,
      "data_type" : 0,
      "status": 0,
      "applications": ["$app$"]
},
"auth":"$auth$",
"id":2
}

The following method will list all the availble json files in specified artifact directory and loops through each file for manipulation.

     /**
     * Registers configuration elements at the zabbix server.
     *
     * @param category    - the type of the items
     * @param items       - name of the item to be registered
     * @param delExisting - the parameter to determine if the user need to delete the existing items
     * @return String - A String containing information about all unsuccessful operations
     */
public void registerConfigurationElements(String category, String[] items, boolean delExisting) {

    deleteExisting = delExisting;
    final String basePath = "conf" + File.separator + "zabbix" + File.separator;

    // Finding the path to Item templates
    File dirItemPath = new File(basePath + category + File.separator + "items");
    if (!dirItemPath.exists()) {
        throw new IllegalArgumentException("Item template directory for type : " + category + " does not exists");
    }

    File dirItems = new File(dirItemPath.getAbsolutePath());
    for (File file : dirItems.listFiles()) {
        int index = file.getName().lastIndexOf(".");
        String extension = file.getName().substring(index + 1);
        if ("json".equals(extension)) {
            processFile(file.getAbsolutePath(), items,
                file.getName().replaceAll(".json", ""), true);
        }
    }

    //find the path to Trigger templates
    File dirTriggerPath = new File(basePath + category + File.separator + "triggers");
    if (!dirTriggerPath.exists()) {
        throw new IllegalArgumentException("Trigger templates for type: " + category + " does not exists");
    }

    File dirTriggers = new File(dirTriggerPath.getAbsolutePath());
    for (File file : dirTriggers.listFiles()) {
        int index = file.getName().lastIndexOf(".");
        String extension = file.getName().substring(index + 1);
        if ("json".equals(extension)) {
            processFile(file.getAbsolutePath(), items,
                file.getName().replaceAll(".json", ""), false);
        }
     }

    // Finding the path to Graph templates
    File dirGraphPath = new File(basePath + category + File.separator + "graphs");
    if (!dirGraphPath.exists()) {
        throw new IllegalArgumentException("Graph templates directory for type : " + category + " does not exists");
    }

    File dirGraphs = new File(dirGraphPath.getAbsolutePath());
    for (File file : dirGraphs.listFiles()) {
        int index = file.getName().lastIndexOf(".");
        String extension = file.getName().substring(index + 1);
        if ("json".equals(extension)) {
            createGraph(file.getAbsolutePath(), items,
                file.getName().replaceAll(".json", ""));
        }
    }
}

If you carefully went through the json template above json template you would have probably noticed some variables wrapped in dollar ($) sign. These variable are replaced by the program.

Bellow is the createGraph method which is used for creating graphs at the zabbix server, but the method for creating items and triggers is not given as functionality is very much the same. Also I have omitted code lines which re-authenticate a if the zabbix session times out, solely for the purpose of maintaining the simplicity of the post.

    private void createGraph(String path, String items[], String fileName) {

        for (String item : items) {
            String request = readFile(path);
            request = request.replace("$filename$", fileName);
            request = request.replace("$id$", item);
            request = request.replace("$auth$", auth);

            try {
                HashMap requestNode = mapper.readValue(request, HashMap.class);
                ArrayList params = (ArrayList) requestNode.get("params");
                HashMap graphItems = (HashMap) params.get(0);
                ArrayList graphItemsList = (ArrayList) graphItems.get("gitems");
                HashMap itemId = (HashMap) graphItemsList.get(0);
                String itemIdList = (String) itemId.get("itemid");
                itemId.remove("itemid");
                itemId.put("itemid", map.get(itemIdList));
                String reqMssg = mapper.writeValueAsString(requestNode);
                postAndGet(reqMssg);

            } catch (EOFException e) {
                logger.warn("Empty json file at : {} - Exception : {}", path, e);
                uiRegisterElemMessage.append("Empty json file at ").append(path).append(" ").append(e).append("\n");

            } catch (IOException e) {
                logger.warn("Could not create graph for item : {} at path : {}", item, path);
                uiRegisterElemMessage.append("Could not create graph for item ").append(item).append(" at path ").
                    append(path).append("\n");
            }
        }
    }

As you can see working with Zabbix api is not difficult. The tedious part is sending requests to the server and processing received responses.

Running the Zabbix Server

The Zabbix server should be invoked by executing this command “/etc/init.d/zabbix-server start” before you start monitoring . You can check whether the Zabbix server is running from the front end GUI.

How ever if the GUI says “NO” have a look at the zabbix server log file at /var/log/zabbix-server/zabbix_server.log. If the log message some what similar to “Access denied for user ‘zabbix’@’localhost’ (using password: YES)”, there is a conflict between the zabbix database password and the passwords saved in following files.

/etc/zabbix/dbconfig.php

/etc/zabbix/zabbix_server.conf

Make sure that all three passwords are the same. The password for the zabbix database can reset by typing “set password for ‘zabbix’@’localhost’ = PASSWORD(‘yourPassword’); ” on the mysql prompt.

Advertisements

4 responses to this post.

  1. Posted by Michael Chan on October 6, 2011 at 6:28 am

    Hi Amindri,
    I’m a university student at the University of Technology in Sydney.
    I’m currently working on a Zabbix iOS mobile application.

    I’m glad to see your program was successfully integrated into the Uconsole. So it seems like you know your way around the Zabbix API. So I would also like to ask for some guidance from yourself.

    I have manged to authenticate and change the parameters of the user using the user.updateProfile method, however I’m having some difficulty with extracting data for the events, triggers and hosts.

    I wish to ask for some assistance in how I may construct the correct the method calls and filtering parameters, because currently I am only receiving empty arrays or an error -32700. You will not be asked any specific coding questions relating to objective C.

    Please email me if you wish to assist me.
    Kind regards,
    Michael

    Reply

    • Posted by amindri udugala on October 6, 2011 at 7:47 am

      Hi Michael,
      I suppose that you you used the .get() method call in Zabbix to retrieve data.

      The following should work on hosts
      {
      “jsonrpc”:”2.0″,
      “method”:”host.get”,
      “params”:{
      “output”:”extend”,
      “filter”:{
      “host”:[“Zabbix-server”,”Zabbix-server TEST”] <<- here you can specify the names of hosts that u want to receive data from
      }
      },
      "auth":"received authentication token after you called the authenticate method call",
      "id":2
      }

      Hope this helps. If you can post the original method call which you wrote I can help you better

      Reply

  2. Hi Amindri,

    I’m a newbie into zabbix and API itself. Could you please help me with my case? I want to display Zabbix’s Map and Graph externally (outside of zabbix) on my website (using php).

    Could you please share the example code to make it possible?

    Thanks.

    Adi

    Reply

    • Posted by amindri udugala on March 1, 2012 at 1:58 pm

      Hi Adi,

      I have done the same thing here but using java. If I summarise what I have done

      1. Authenticate at the Zabbix server and get the authentication token. (for this you have to issue a post request to the Zabbix server and then a GET request to receive the authentication token. Since you are using PHP you have to use some package to issue REST requests. I think PHP has that ability inbuilt. I’m not familiar with php)

      2. Once you get the authentication token use that to retreive MAP data from the Zabbix Server. A sample POST request to receive MAP data would be as follows

      {“jsonrpc”:”2.0″,
      “method”:”graph.post”,
      “params”:[{
      $graphid$
      “gitems”:[{
      “itemid”:”jmx[org.adroitlogic.ultraesb.detail:Type=FileCaches,Name=$id$][Details.availableForUse]”,
      “drawtype”:”0″,
      “sortorder”:”0″,
      “color”:”999900″,
      “yaxisside”:”0″,
      “calc_fnc”:”2″,
      “type”:”0″,
      “periods_cnt”:”5″
      }],
      “name”:”$filename$-$id$”,
      “width”:”900″,
      “height”:”200″,
      “yaxismin”:”0.0000″,
      “yaxismax”:”3.0000″,
      “templateid”:”0″,
      “show_work_period”:”1″,
      “show_triggers”:”1″,
      “graphtype”:”0″,
      “show_legend”:”0″,
      “show_3d”:”0″,
      “percent_left”:”0.0000″,
      “percent_right”:”0.0000″,
      “ymin_type”:”0″,
      “ymax_type”:”0″,
      “ymin_itemid”:”0″,
      “ymax_itemid”:”0″
      }],
      “auth”:”$auth$”,
      “id”:2
      }

      The variables $graphid$ and $auth$ are replaced inside the java function. (Authentication token changes at each issue)

      Hope this helps 🙂
      Let me know if you have any thing else to be clarified.

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: