InfluxDB
InfluxDB is an open-source, distributed, time-series database.
Official site: https://influxdata.com
InfluxDB is an open-source, distributed, time-series database.
Official site: https://influxdata.com
I installed InfluxDB on CentOS 7 64-bit virtual machine in june 2016. At that time the actual version of InfluxDB was 0.12.0
We'll use yum repo to instal InfluxDB. Create new repo file:
$ vi /etc/yum.repos.d/influxdb.repo
[influxdb]
name = InfluxDB Repository - RHEL \$releasever
baseurl = https://repos.influxdata.com/rhel/\$releasever/\$basearch/stable
enabled = 1
gpgcheck = 1
gpgkey = https://repos.influxdata.com/influxdb.key
Then install InfluxDB with yum:
$ yum install influxdb
Start InfluxDB service:
$ service influxdb start
Or:
$ systemctl start influxdb
Before InfluxDB can be used, make sure that tcp ports 8083 and 8086 are enabled in firewall to accept connections. Port 8083 is used for administration and port 8086 is used for client-server communication via http api.
Here is example how to configure iptables in CentOS:
$ vi /etc/sysconfig/iptables
Add the following lines:
...
-A INPUT -m state --state NEW -m tcp -p tcp --dport 8083 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 8086 -j ACCEPT
...
Restart the firewall:
$ /sbin/service iptables restart
CLI is the simplest way to start learning InfluxDB.
Start InfluxDB interactive command line interface:
$ influx
Connected to http://localhost:8086 version 0.12.0
InfluxDB shell 0.12.0
>
For InfluxQL commands see next article.
To exit the CLI type:
> exit
Here are some most useful CLI arguments:
$ influx -version
$ influx -execute 'SHOW DATABASES'
$ influx -execute 'SELECT * FROM "cpu" LIMIT 3' -database="testdb"
$ influx -execute 'SELECT * FROM "cpu" LIMIT 3' -database="testdb" -format=json -pretty
Basics of InfluxDB Query Language.
See below for Advanced
Create database:
> CREATE DATABASE testdb
> SHOW DATABASES
> USE testdb
Insert and read data:
> INSERT cpu,system=serverA,user=root value=0.64
> SELECT system, user, value FROM cpu
> SELECT * FROM cpu
> INSERT temperature,system=mycentos,type=server external=25,internal=37
> SELECT * FROM temperature
> SHOW RETENTION POLICIES
> SHOW MEASUREMENTS
> SHOW SERIES
> SHOW TAG KEYS
> SHOW TAG VALUES
> SHOW FIELD KEYS
> SHOW TAG KEYS FROM cpu
Delete data:
> DROP MEASUREMENT cpu
> DROP SERIES
> DROP DATABASE testdb
Syntax of InfluxQL
table,key=value,key2=value2 key=value,key2=value2 timestamp
|-M-|,|--------TAGS-------| |-------FIELDS------| |--time-|
First parameter is table name, also called measurement, followed by tags as key-value pairs, separated by coma (','). Then comes whitespace and then fields as key-value pairs, separated by coma (','). Then whitespace and at the end there is a timestamp in nanoseconds. Timestamp is optional; if not provided system time will be inserted automatically.
InfluxDB Web GUI is reachable with web browser on:
InfluxDB offers HTTP API to communicate with external systems and clients. HTTP clients are very easy to implement in any programming language. Response is returned in JSON format.
HTTP API provides 3 endpoints:
/ping - check database status
/query - query data
/write - write data to database
Here are some examples of InfluxDB queries made with different tools or programming languages.
InfluxDB queries with curl
Check database status (http get)
$ curl -sl -I 192.168.1.115:8086/ping
Create database
$ curl -POST http://192.168.1.115:8086/query --data-urlencode "q=CREATE DATABASE testdb"
Insert data
$ curl -i -X POST 'http://192.168.1.115:8086/write?db=testdb' --data-binary 'temp,sensor=T1,region=room value=23'
# with timestamp
$ curl -i -X POST 'http://192.168.1.115:8086/write?db=testdb' --data-binary 'temp,sensor=T1,region=room value=23 1422568543702900257'
Query data
$ curl -GET 'http://192.168.1.115:8086/query?pretty=true' --data-urlencode "db=testdb" --data-urlencode "q=SELECT \"value\" FROM \"cpu\" WHERE \"system\"='centos7'"
field type conflict
The problem is with inconsistent data types.
If you get the following error response (HTTP error code 400):
{"error":"field type conflict: input field \"value\" on measurement \"cpu\" is type float64, already exists as type integer"}
It means that data types in request and in database do not match.
InfluxDB determines field value type when first point is inserted. Once data type is set, all further points must use the same type as inserted points.
InfluxDB automatically treats numbers as float64 type unless stated otherwise. To overcome the problem with different types, specify all numbers as float, even whole numbers (10 is 10.0).
To insert a measurement into InfluxDB in Python, you need to write a http client that sends POST request to InfluxDB HTTP API.
Here is piece of python code that initiates http connection with the InfluxDB server, prepares POST request and prints returned HTTP error code.
Remark: InfluxDB returns error code 204 if measurement was successfully inserted.
#!/usr/bin/env python
import httplib
httpServ = httplib.HTTPConnection("192.168.1.115", 8086)
httpServ.connect()
stringValue = str(22)
httpServ.request('POST', '/write?db=myDatabase', 'temperature,sensor=T1 value=' + stringValue)
response = httpServ.getresponse()
print response.status
httpServ.close()
Send HTTP GET request to query data
public void sendGet() throws Exception {
String url = "http://192.168.1.115:8086/query?db=testdb&"
+ "q=SELECT value FROM cpu WHERE system='centos7'";
url = url.replace(" ", "%20");
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("GET");
int responseCode = con.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
System.out.println(response.toString());
}
Send HTTP POST request to insert data (response code 204 means successfully inserted)
public void sendPost() throws Exception {
String url = "http://192.168.1.115:8086/write?db=testdb";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("POST");
String urlParameters = "cpu,system=centos7,user=matjaz value=150.00";
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.writeBytes(urlParameters);
wr.flush();
wr.close();
int responseCode = con.getResponseCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + urlParameters);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
System.out.println(response.toString());
}
Advanced use of InfluxDB Query Language.
SELECT statement with conditions:
> SELECT system, user, value FROM cpu
> SELECT * FROM cpu
> SELECT * FROM cpu WHERE system='serverA'
> SELECT "system","value" FROM cpu WHERE "user"='root' LIMIT 10
> SELECT * FROM "cpu" WHERE time > now() - 7d
> SELECT * FROM "cpu" WHERE "system" = 'serverA' AND "value" > 75
Grouping data:
> SELECT MEAN("value") FROM "cpu" GROUP BY "system"
> SELECT COUNT("value") FROM "cpu" WHERE time >= '2016-08-19T00:00:00Z' AND time <= '2016-08-27T17:00:00Z' AND "system"='serverA' GROUP BY time(3d)
> SELECT MEAN("value") FROM "cpu" WHERE time > now() - 2w GROUP BY "system",time(3h)
> SELECT MEAN("value") FROM "cpu" WHERE time > now() - 2w GROUP BY "system",time(3h) fill(-1)
Relocating data:
> SELECT "value" INTO "cpu_copy" FROM "cpu" WHERE "system" = 'serverA'
> SELECT MEAN("value"),max("value") INTO "cpu_copy" FROM "cpu" WHERE time > now() - 2w GROUP BY "system",time(3h)
Ordering data:
> SELECT "value" FROM "cpu" WHERE "system" = 'serverA' ORDER BY time DESC LIMIT 5
For many other features and examples read official InfluxDB documentation at: https://influxdata.com