GridDB quick start guide
Revision: 2200
Table of Contents
- 1 Introduction
- 2 System design and construction
- 3 Operations
- 4 Annexes
- 5 Trademarks
1 Introduction
1.1 Aim and composition of this document
This manual describes the basic operating procedure of GridDB.
It is written for system designer and system administrators responsible for GridDB system's construction and operation management respectively.
Two main subjects are covered in this manual:
-
System design and setup
- Basic setup of GridDB post-installation is explained in this section.
-
Operation
- Basic operation such as starting and stopping of GridDB, operation management, and troubleshooting are explained in this section.
1.2 What is GridDB?
1.2.1 Overview
GridDB is a distributed NoSQL database that manages a group of data called a row, which contains a key and multiple values. There are currently 2 commercial version of GridDB in offer.
- GridDB Standard Edition
- GridDB Advanced Edition
GridDB Advanced Edition adds SQL processing engine on top of the high performance offered in GridDB Standard Edition.
Even though the library used during application development is different, system construction and operation remains the same.
GridDB’s merits are as follows:
-
Allows data management to be carried out in memory at a high speed.
- Groups of rows are stored in memory, allowing high-speed updates and searches to be carried out.
-
Allows in-memory processing to be carried out and its capacity to be increased.
- Capacity can be increased by dispersing and placing data in multiple machines. Furthermore, data management using disks is also possible (this operation is not covered in this manual). As a result, even when operating with individual nodes, the capacity can be increased without being limited by the memory size.
-
Able to achieve high availability.
- Data is replicated within a cluster, thus allowing uninterrupted process in the case of a failure in any of the cluster’s node. In addition, each node continuously updates data to the disk, allowing restoration of data when a failure occurs.
-
Can scale out up to 1000 units.
- In transactions involving container as the unit, high scalability is achieved by improving the degree of parallelism of the cluster process.
-
Does not require manual cluster management.
- Communications are carried out using a decentralized protocol among mutual nodes, with cluster control carried out autonomously by GridDB itself.
-
Supports non-standard data used in social infrastructure.
- It supports non-standard data such as time series data and spatial data etc. that is used in social infrastructure.
- In the GridDB Advanced Edition, ODBC/JDBC interfaces are supported.
1.2.2 Features
The features of GridDB are summarized below.
Requirements | Description |
---|---|
[Basic requirements] | |
Large capacity (petabyte class) | In order to achieve both high speed and high capacity, data is stored in memory and SSD devices. |
High speed (in-memory) | In-memory processing |
High scalability | The number of servers can be increased up to 1000 units |
High availability | Availability is further improved by replicating data to HDD in multiple servers. |
High autonomy | Automation of data replication and data placement balancing. Server expansion can be easily done online. |
Operation control functions | Monitoring, security, backup etc. |
[Social infrastructure requirements] | |
Time series data | A lossless compression function that could reduce the data size yet keeping the original features unchanged is provided. |
Spatial data type | The index can be tuned to increase the search speed. 2D and 3D data type are supported. |
Consistency guarantee | Supports ACID transactions within the same container. |
1.3 Explanation of terms
The following table explains the GridDB related terms commonly found in this manual.
Term | Meaning |
---|---|
Node | Individual server process to perform data management in GridDB. |
Cluster | Single or a set of nodes that perform data management together in an integrated manner. |
Partition | Logical area where data is stored. It exists only within GridDB, and not visible directly to the user. |
Row | A chunk of data managed in GridDB, which consists of a key and multiple values. |
Container (table) | Data structure serving as an I/F with the user. Container to manage a set of rows. It is called a container when operating with NoSQL I/F, and a table when operating with NewSQL I/F. 2 data types exist, collection (table) and timeseries container (timeseries table). |
Collection (table) | A type of container that manages rows containing general key. |
Timeseries container (timeseries table) | A type of container that manages rows containing time-series key. It has a special function to handle time-series data. |
Master node | The node that performs cluster management. |
Follower node | All other nodes in the cluster other than the master node. |
Owner node | Within the replicated containers, it records the location of the master container. |
Backup node | Within the replicated containers, it records the location of the containers’ replica. |
2 System design and construction
The basic system design and construction flow is shown in this chapter.
The design and construction of GridDB nodes and clusters are carried out as follows.
- Check that the required resources are sufficient
- Perform node installation and setup
- Set the environment-dependent parameters
- Set the tuning parameters
- Distribute the definition file to each node
For client setting, please refer to the following item.
2.1 Checking the required resources
GridDB is a scale-out database that unlike a conventional DB, careful system design and sizing is not required in order to achieve a non-stop operation. However, the following points should be considered as guidelines in the initial system design.
- Memory usage
- Number of nodes in a cluster
- Disk usage
The estimation method is explained in sequence below.
However, functions to increase the capacity by using external storage devices such as SSDs etc. have not been considered in calculating the memory size below. Please check with our service desk for estimation if these functions are used.
2.1.1 Total memory usage
Predict the amount of data to be stored in the container and then estimate the memory usage.
First, predict the amount of data to be stored in the application. Estimate the following:
- Data size of row
- Number of rows to be registered
Next, estimate the memory required to store those estimated data. The formula to calculate the memory usage differs by container type, as shown below.
-
Collection
- Memory usage (byte) = 600 × no. of containers + 1.8 × row size × no. of all rows + 24 × no. of all rows × no. of indices
-
Time-series container
- Memory usage (byte) = 700 × no. of containers + 1.5 × row size × no. of all rows + 24 × no. of all rows × no. of indices
Perform a similar estimation for all containers created and used in the application. The total sum becomes the amount of memory required in the GridDB cluster.
- Total memory usage = the sum of memory usage in all containers
However, please consider this number as the minimum requirement, as the memory usage also depends on update frequency.
2.1.2 No. of nodes constituting a cluster
Estimate the required no. of nodes used in GridDB. In the example below, it is assumed that one node is executed in one machine.
First, make an assumption of the memory required per machine.
- Memory size of machine
In addition, make an assumption of the no. of replicas to create. The no. of replicas is given as a GridDB configuration value.
- No. of replicas
Default no. of replicas is 2.
- No. of nodes = (Total memory usage ÷ machine memory size) × no. of replicas (units)
However, please consider this as the minimum requirement as it is preferable to have a greater number of spare units to take into account the load distribution and availability improvement.
2.1.3 Disk usage
Estimate the size of the file to be created in GridDB, and the disk capacity required for the machine to execute a node. Two types of files should be created; a checkpoint file and a transaction log file.
The memory usage of individual node is determined as follows.
- Individual memory usage = (total memory usage × no. of replicas) ÷ no. of nodes (byte)
The size of the checkpoint file is estimated as follows based on this numerical value.
- File size = Individual memory usage × 2 (byte)
In addition, as the transaction log file size is dependent on the update frequency of the application, the following data is thus predicted.
- Row update frequency (times/sec)
Furthermore, the checkpoint interval is assumed. Checkpoint interval is given as a GridDB configuration value.
- Checkpoint interval
Default value of checkpoint interval is 60 sec (1 minute).
The transaction log file size is estimated as follows based on these numerical values.
- File size = row data size × row update frequency × checkpoint interval (byte)
The individual disk usage is estimated as follows.
- Individual disk usage = Transaction log file size + checkpoint file size
[Points to note]
- The size of checkpoint file expands depending on the data capacity. However, please note that once it has expanded, the file size will not be reduced even if some of data in the container or row get deleted. Empty space can be re-used after data is deleted.
2.2 Installation and setup (node)
This chapter explains the installation of a node in a single machine. See the chapter on operations for the cluster configuration.
2.2.1 Environmental check
Check that the version is RHEL 6.2/6.3/6.4/6.5/6.6/6.7/6.8/6.9/7.2/7.3/7.4/7.5, or CentOS 6.2/6.3/6.4/6.5/6.6/6.7/6.8/6.9/7.2/7.3/7.4/7.5.
$ cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core)
[Points to note]
-
When choosing the package group during OS installation, please choose the following version or higher.
- Basic Server
2.2.2 Installing a node
The following 3 RPM packages are needed when installing a GridDB node. Place these packages anywhere in the machine.
Package name | File name | Description |
---|---|---|
griddb-xx-server | griddb-xx-server-X.X.X-linux.x86_64.rpm | The start and other commands for the GridDB node module and server are included. |
griddb-xx-client | griddb-xx-client-X.X.X-linux.x86_64.rpm | Operating commands except start node are included. |
griddb-xx-docs | griddb-xx-docs-X.X.X-linux.x86_64.rpm | GridDB manuals and program samples are included. |
*: X.X.X is the GridDB version
Install using the rpm command as a root user.
$ su # rpm -Uvh griddb-xx-server-X.X.X-linux.x86_64.rpm Preparing... ########################################### [100%] User gsadm and group gridstore have been registered. GridDB uses new user and group. 1:griddb-xx-server ########################################### [100%] # rpm -Uvh griddb-xx-client-X.X.X-linux.x86_64.rpm Preparing... ########################################### [100%] User and group has already been registered correctly. GridDB uses existing user and group. 1:griddb-xx-client ########################################### [100%] # rpm -Uvh griddb-xx-docs-X.X.X-linux.x86_64.rpm Preparing... ########################################### [100%] 1:griddb-xx-docs ########################################### [100%]
When you install the package, the following group and user are created in the OS. This OS user is set as the operator of GridDB.
Group | User | Home directory |
---|---|---|
gridstore | gsadm | /var/lib/gridstore |
The following environment variables are defined in this gsadm user.
Environment variables | Value | Meaning |
---|---|---|
GS_HOME | /var/lib/gridstore | gsadm/GridDB home directory |
GS_LOG | /var/lib/gridstore/log | Event log file output directory |
[Points to note]
- These environment variables are used in the operating commands described later.
-
The password of the gsadm user has not been set. With the root user privilege, please set the password appropriately.
- Some of the functions of the operation tools may be necessary.
In addition, when the GridDB node module is installed, services that are executed automatically upon startup of the OS will be registered.
Service name | Run level |
---|---|
gridstore | 3,4,5 |
The service registration data can be checked with the following command.
# /sbin/chkconfig --list | grep gridstore gridstore 0:off 1:off 2:off 3:on 4:on 5:on 6:off
The GridDB node will be started automatically by this service during OS startup.
[Points to note]
- Service will not start automatically immediately after installation.
To stop auto startup of a service, use the command below.
# /sbin/chkconfig gridstore off
To start the service, it is necessary to set the startup configuration file (/etc/sysconfig/gridstore/gridstore.conf). See the chapter on services in “GridDB operation control guide” (GridDB_OperationGuide.html) for details of the services.
2.2.3 Post-installation checks
Check the directory configuration of the installed GridDB node.
First, check that the GridDB home directory, and related directory and files have been created.
GridDB home directory
/var/lib/gridstore/ # GridDB home directory admin/ # integrated operational management GUI home directory backup/ # backup directory conf/ # definition file directory gs_cluster.json # cluster definition file gs_node.json # node definition file password # user definition file data/ # database file directory log/ # log directory
Confirm with the following command.
$ ls /var/lib/gridstore/ admin backup conf data log
Next, check that the installation directory has been created.
Installation directory
/usr/griddb/ # installation directory Fixlist.pdf # revision record Readme.txt # release instructions bin/ # operation command, module directory conf/ # definition file sample directory docs/ # document directory etc/ lib/ # library directory license/ # license directory prop/ # configuration file directory web/ # integrated operational management GUI file directory
Confirm with the following command.
$ ls /usr/griddb/ Fixlist.pdf Readme.txt bin conf etc lib license prop web
All documents have been compressed into a single ZIP file. Decompress and refer to the documents where appropriate as shown below.
$ cd /usr/griddb/docs $ unzip griddb-documents-X.X.X.zip
Lastly, confirm the version of the server module installed with the following command.
$ gsserver --version GridDB version X.X.X build XXXXX ***** Edition
[Points to note]
The following files are created when GridDB is operated according to the following procedure.
[Database file]
/var/lib/gridstore/ # GridDB home directory data/ # database file directory gs_log_n_m.log # log file to record the transaction logs (n, m are numbers) gs_cp_n_p.dat # checkpoint file to record data regularly (n, p are numbers)
[Log file]
/var/lib/gridstore/ # GridDB home directory log/ # log directory gridstore-%Y%m%d-n.log # event log file gs_XXXX.log # operation tool log file
The directory where these files are created can be changed by the parameter settings in the node definition file.
*: gs_XXXX is an operation tool name. (Example: gs_startnode.log)
2.2.4 Setting up an administrator user
Administrator privilege is used for authentication related matter within the nodes and clusters. Creation date of administrator user is saved in the user definition file . The default file is as shown below.
- /var/lib/gridstore/conf/password
The default user below exists immediately after installation.
User | Password | Example of proper use |
---|---|---|
admin | admin | For authentication of operation administrator user, operation commands |
system | manager | For authentication of application user, client execution |
Administrator user information including the above-mentioned default users can be changed using the user administration command in the operating commands.
Command | Function |
---|---|
gs_adduser | Add an administrator user |
gs_deluser | Delete an administrator user |
gs_passwd | Change the password of administrator user |
Change the password as shown below when using a default user. The password is encrypted during registration.
$ gs_passwd admin Password: (enter password) Retype password: (re-enter password)
When adding a new administrator user except a default user, the user name has to start with gs#.
One or more ASCII alphanumeric characters and the underscore sign “_” can be used after gs#.
An example on adding a new administrator user is shown below.
$ gs_adduser gs#newuser Password: (enter password) Retype password: (re-enter password)
[Points to note]
- A GridDB administrator user is different from the OS user gsadm created during installation.
- A change in the administrator user information using a user administration command becomes valid when a node is restarted.
- In order to use it for authentication purposes in the client, the same user data needs to be registered in all the nodes. Copy the user definition file and make sure the same user data can be referred to in all the nodes.
- Execute the operating command as a gsadm user.
[Memo]
- See “GridDB operation control guide” (GridDB_OperationGuide.html) for details of the user management commands.
2.3 Setting the environment-dependent parameters
After installation, the following settings are necessary in order to operate GridDB.
- Network environment settings
- Cluster name settings
GridDB settings are configured by editing 2 types of definition files.
- Cluster definition file (gs_cluster.json)
- Node definition file (gs_node.json)
The cluster definition file defines the parameters that are common in the entire clusters.
The node definition files define the parameters for the different settings in each node.
These definition file samples are installed as follows.
/usr/griddb/ # installation directory conf/ # definition file directory gs_cluster.json.tmpl # cluster definition file sample gs_node.json.tmpl # node definition file sample
In a new installation, the same files are also placed in the conf directory under the GridDB home directory.
/var/lib/gridstore/ # GridDB home directory conf/ # definition file directory gs_cluster.json # (edited) cluster definition file gs_node.json # (edited) node definition file
During operations, edit these definition files.
[Points to note]
- When the GridDB version is upgraded, compare the newly installed sample with these definition files to adequately reflect the parameters added.
- A cluster definition file defines the parameters that are common in the entire clusters. As a result, the settings must be the same in all of the nodes in the cluster. Nodes with different settings will get an error upon joining the cluster and prevented from participating in the cluster. Further details will be explained in the later chapter.
2.3.1 Network environment settings (essential)
First, set up the network environment.
An explanation of the recommended configuration method in an environment that allows a multicast to be used is given below. In an environment which does not allow a multicast to be used, or an environment in which communications between fellow nodes cannot be established in a multicast, a cluster configuration method other than the multicast method has to be used. See "GridDB operation control guide" (GridDB_OperationGuide.html) for the details.
The configuration items can be broadly divided as follows.
- (1) Address information serving as an interface with the client
- (2) Address information for cluster administration and processing
- (3) Address information serving as an interface with the JDBC client (GridDB Advanced Edition only)
Although these settings need to be set to match the environment, basically default settings will also work.
However, an IP address derived in reverse from the host name of the machine needs to be an address that allows it to be connected from the outside regardless of whether the GridDB cluster has a multiple node configuration or a single node configuration.
Normally, this can be set by stating the host name and the corresponding IP address in the /etc/hosts file.
/etc/hosts setting
First, check with the following command to see whether the setting has been configured. If the IP address appears, it means that the setting has already been configured.
$ hostname -i 192.168.11.10
The setting has not been configured in the following cases.
$ hostname -i hostname: no address corresponding to name
In addition, a loopback address that cannot be connected from the outside may appear.
$ hostname -i 127.0.0.1
If the setting has not been configured or if a loopback address appears, use the following example as a reference to configure /etc/hosts. The host name and IP address, and the appropriate network interface card (NIC) differ depending on the environment.
-
Check the host name and IP address.
$ hostname GS_HOST $ ip route | grep eth0 | cut -f 12 -d " " | tr -d "\n" 192.168.11.10
-
Add the IP address and corresponding host name checked by the root user to the /etc/hosts file.
192.168.11.10 GS_HOST
-
Check that the settings have been configured correctly.
$ hostname -i 192.168.11.10
(*)If the displayed setting remains the same as before, it means that a setting higher in priority is given in the /etc/hosts file. Change the priority order appropriately.
Proceed to the next setting after you have confirmed that /etc/hosts has been configured correctly.
(1) Address information serving as an interface with the client
In the address data serving as an interface with the client, there are configuration items in the node definition file and cluster definition file.
Node definition file
Parameters | Data type | Meaning |
---|---|---|
/transaction/serviceAddress | string | Reception address of transaction process |
/transaction/servicePort | int | Reception port of transaction process |
/system/serviceAddress | string | Connection address of operation command |
/system/servicePort | int | Connection port of operation command |
The reception address and port of transaction processes are used to connect individual client to the nodes in the cluster, and to request for the transaction process from the cluster. This address is used when configuring a cluster with a single node, but in the case where multiple nodes are present through API, the address is not used explicitly.
The connection address and port of the operating command are used to specify the process request destination of the operation command, as well as the repository information of the integrated operation control GUI.
These reception/connection addresses need not be set so long as there is no need to use/separate the use of multiple interfaces.
Cluster definition file
Parameters | Data type | Meaning |
---|---|---|
/transaction/notificationAddress | string | Interface address between client and cluster |
/transaction/notificationPort | int | Interface port between client and cluster |
A multi-cast address and port are specified in the interface address between a client and cluster. This is used by a GridDB cluster to send cluster information to its clients and for the clients to send processing requests via the API to the cluster. See the description of GridStoreFactory class/method in “GridDB API reference” (GridDB_API_Reference.html) for details.
It is also used as a connection destination address of the export/import tool, or as repository data of the integrated operation control GUI.
(2) Address information for cluster administration and processing
In the address data for a cluster to autonomously perform cluster administration and processing, there are configuration items in the node definition file and cluster definition file. These addresses are used internally by GridDB to exchange the heart beat (live check among clusters) and information among the clusters. These settings are not necessary so long as the address used is not duplicated with other systems on the same network or when using multiple network interface cards.
Node definition file
Parameters | Data type | Meaning |
---|---|---|
/cluster/serviceAddress | string | Reception address used for cluster administration |
/cluster/servicePort | int | Reception port used for cluster administration |
Cluster definition file
Parameters | Data type | Meaning |
---|---|---|
/cluster/notificationAddress | string | Multicast address for cluster administration |
/cluster/notificationPort | int | Multicast port for cluster administration |
-
Although a synchronization process is carried out with a replica when the cluster configuration is changed, a timeout time can be set for the process.
- /sync/timeoutInterval
[Points to note]
- An address or port that is not in use except in GridDB has to be set.
- The same address can be set for the node definition file gs_node.json /transaction/serviceAddress, /system/serviceAddress, and /cluster/serviceAddress for operations to be performed. If a machine has multiple network interfaces, the bandwidth can be increased by assigning a separate address to each respective interface.
The following settings are applicable in the GridDB Advanced Edition only.
(3) Address information serving as an interface with the JDBC client
In the address data serving as an interface with the JDBC/ODBC client, there are configuration items in the node definition file and cluster definition file.
Node definition file
Parameters | Data type | Meaning |
---|---|---|
/sql/serviceAddress | string | Reception address for JDBC/ODBC client connection |
/sql/servicePort | int | Reception port for JDBC/ODBC client connection |
The reception address and port of JDBC/ODBC client connection are used to connect JDBC/ODBC individual client to the nodes in the cluster, and to access the cluster data in SQL. This address is used when configuring a cluster with a single node, but in the case where multiple nodes are present through API, the address is not used explicitly.
Cluster definition file
Parameters | Data type | Meaning |
---|---|---|
/sql/notificationAddress | string | Address for multi-cast distribution to JDBC/ODBC client |
/sql/notificationPort | int | Multicast port to JDBC/ODBC client |
The address and port used for multicast distribution to a JDBC/ODBC client are used for the GridDB cluster to notify the JDBC/ODBC client of cluster data, and to access the cluster data in SQL with the JDBC/ODBC client.
Refer to Annex Parameter list for the other parameters and default values.
2.3.2 Cluster name settings (essential)
Set the name of the cluster to be composed by the target nodes in advance. The name set will be checked to see if it matches the value specified in the command to compose the cluster. As a result, this prevents a different node and cluster from being composed when there is an error in specifying the command.
The cluster name is specified in the following configuration items of the cluster definition file.
Cluster definition file
Parameters | Data type | Meaning |
---|---|---|
/cluster/clusterName | string | Name of cluster to create |
[Points to note]
- Node failed to start with default value ("").
- A unique name on the sub-network is recommended.
- A cluster name is a string composed of 1 or more ASCII alphanumeric characters and the underscore “_”. However, the first character cannot be a number. The name is also not case-sensitive. In addition, it has to be specified within 64 characters.
2.4 Setting the tuning parameters
The main tuning parameters are described here. These parameters are not mandatory but affect the processing performance of the cluster.
2.4.1 Parameter related to update performance
GridDB creates a transaction log file and checkpoint file for persistency purposes. As data written in these files has an impact on the update performance, the method of creation can be changed by the following parameters. However, the disadvantage is that the possibility of data being lost during a failure is higher.
The related parameters are shown below.
Node definition file
Parameters | Data type | Meaning |
---|---|---|
/dataStore/persistencyMode | string | Persistency mode |
/dataStore/logWriteMode | int | Log write mode |
The persistency mode specifies whether to write data to a file during a data update. The log write mode specifies the timing to write data to the transaction log file.
Either one of the values below can be set in the persistency mode.
- "NORMAL"
-
"KEEP_ALL_LOGS"
"NORMAL" writes data to the transaction log file and checkpoint file every time there is an update. A transaction log file which is no longer required will be deleted by a checkpoint. The write timing of "KEEP_ALL_LOGS" is the same as "NORMAL" but all transaction log files are retained. Default value is "NORMAL".
[Points to note]
- When performing a differential backup, set the persistency mode to "NORMAL".
Either one of the values below can be set in the log write mode.
- 0: SYNC
- An integer value of 1 or higher: DELAYED_SYNC
In the "SYNC" mode, log writing is carried out synchronously every time an update transaction is committed or aborted. In the "DELAYED_SYNC" mode, log writing during an update is carried out at a specified delay of several seconds regardless of the update timing. Default value is "1 (DELAYED_SYNC 1 sec)".
2.4.2 Parameter related to performance and availability
By composing a cluster, GridDB can replicate data in multiple nodes to improve the search performance and availability. As replication of these data has an impact on the update performance, the method of creation can be changed by the following parameters. However, the disadvantage is that the possibility of data being lost during a failure is higher.
The related parameters are shown below.
Cluster definition file
Parameters | Data type | Meaning |
---|---|---|
/transaction/replicationMode | int | Replication mode |
The replication mode specifies how to create a replica. The replication method has to be same for all nodes in the cluster.
- "0": Asynchronous replication
- "1": Semi-synchronous replication
"Asynchronous replication" performs replication without synchronizing the timing of the update process. "Semi-synchronous replication" performs replication synchronously at the timing of the update process timing but makes no appointment at the end of the replication. Default is "0".
2.4.3 Parameter related to access performance immediately after startup
Data perpetuated on a disk etc. can be loaded into the memory at the same time when a node is started (warm start process).
The warm start process can be enabled/disabled by the parameter below.
Node definition file
Parameters | Data type | Meaning |
---|---|---|
/dataStore/storeWarmStart | boolean | Start processing mode |
- false: non-warm start mode
- true: warm start mode
Default is false (disabled).
2.4.4 Other parameters
An explanation of the other parameters is given. See Annex Parameters list for the default values.
Node definition file
Parameters | Data type | Meaning |
---|---|---|
/dataStore/dbPath | string | Database file directory |
/dataStore/backupPath | string | Backup file directory |
/dataStore/storeMemoryLimit | string | Memory buffer size |
/dataStore/concurrency | int | Processing parallelism |
/dataStore/affinityGroupSize | int | No. of data affinity groups |
/checkpoint/checkpointInterval | int | Checkpoint execution interval (unit: sec) |
/system/eventLogPath | string | Output directory of event log file |
/transaction/connectionLimit | int | No. of connections upper limit value |
/trace/[category] | string | Event log output level |
- A database file directory is created when the data registered in the in-memory is perpetuated. In this directory, the transaction log file and checkpoint files are created.
- A backup file directory is created when a backup is executed (further details will be explained in the subsequent chapters). In this directory, the backup file is stored.
- The memory buffer size is used for data management. To set the memory size, simply use numbers and text to specify the memory size and its unit respectively. E.g. “2048MB”.
- Processing parallelism refers to the upper limit value for GridDB to perform I/O processing of the secondary memory device in parallel.
- With regards to data affinity, related data are collected, and the number of groups is specified during layout management.
- Any number from 1 to 64 can be selected when specifying the number of groups. Please note though that the larger the group, the lower the memory utilization efficiency becomes.
- The checkpoint execution interval is the interval to execute an internal checkpoint process regularly (related to the perpetuation of data).
- The output directory of event log is the directory where messages (event message files) related to event data occurring inside a node such as exceptions etc, will be saved into.
- As a rule of thumb, please set the no. of connection upper limit to at least twice the number of expected clients.
- The event log output level is the output level for each category of the event log.
2.5 Distributing the definition file to each node
Both of the user and cluster definition files need to have the same settings in all of the nodes within the GridDB cluster.
Therefore, when configuring a cluster with 2 or more nodes, please follow these steps to set up all the nodes at once. (When configuring a cluster with a single node, the following steps can be skipped.)
- Set up the administrator user settings and environment-dependent parameters on either of the machines installed with nodes.
- Copy the already configured cluster definition file and user definition file to the definition file directory in the other nodes.
- Copy the node definition file too when the configuration setting is common to all of the nodes.
- Configure the nodes with different settings separately. (Network environment settings etc.)
2.6 Installation and setup (client)
This chapter explains the installation procedure of the client library. There are 2 types of client library in GridDB, Java and C. Only the Java version supports the GridDB NewSQL interface.
2.6.1 Environmental check
Check that the version is RHEL 6.2/6.3/6.4/6.5/6.6/6.7/6.8/6.9/7.2/7.3/7.4/7.5, or CentOS 6.2/6.3/6.4/6.5/6.6/6.7/6.8/6.9/7.2/7.3/7.4/7.5.
$ cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core)
[Points to note]
-
When choosing the OS package group during OS installation, please choose the minimum version or lower.
- Software Development WorkStation
The following needs to be installed as a Java language development environment.
- Oracle Java 7/8
- Only 64-bit Java is supported by the GridDB NewSQL interface.
$ java -version java version "1.7.0_79" Java(TM) SE Runtime Environment (build 1.7.0_79-b15) Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)
2.6.2 Installing the client library
The following 4 RPM packages are required for the installation of the GridDB client library. Place the packages anywhere in the machine.
The griddb-xx-newsql package is only available in GridDB Advanced Edition.
Package name | File name | Description |
---|---|---|
griddb-xx-java_lib | griddb-xx-java_lib-X.X.X-linux.x86_64.rpm | Java library is included. |
(/usr/share/java/gridstore.jar) | ||
griddb-xx-c_lib | griddb-xx-c_lib-X.X.X-linux.x86_64.rpm | C header file and library are included. |
(/usr/include/gridstore.h and /usr/lib64/libgridstore.so) | ||
griddb-xx-python_lib | griddb-xx-python-X.X.X-linux.x86_64.rpm | Python library is included. |
griddb-xx-docs | griddb-xx-docs-X.X.X-linux.x86_64.rpm | GridDB manuals and program samples are included. |
griddb-xx-newsql | griddb-xx-newsql-X.X.X-linux.x86_64.rpm | NewSQL interface library is included. |
[Points to note]
- Install the python library package as needed. In the case of the installation, it is necessary to install the C library package beforehand.
Install using the rpm command as a root user.
$ su # rpm -ivh griddb-xx-c_lib-X.X.X-linux.x86_64.rpm Preparing... ########################################### [100%] 1:griddb-xx-c_lib ########################################### [100%] # rpm -ivh griddb-xx-java_lib-X.X.X-linux.x86_64.rpm Preparing... ########################################### [100%] 1:griddb-xx-java_lib ########################################### [100%] # rpm -ivh griddb-xx-python_lib-X.X.X-linux.x86_64.rpm Preparing... ########################################### [100%] 1:griddb-xx-python_lib ########################################### [100%] # rpm -ivh griddb-xx-docs-X.X.X-linux.x86_64.rpm Preparing... ########################################### [100%] 1:griddb-xx-docs ########################################### [100%] # rpm -ivh griddb-xx-newsql-X.X.X-linux.x86_64.rpm Preparing... ########################################### [100%] 1:griddb-xx-newsql ########################################### [100%]
2.6.3 Post-installation checks
After installation, check the directory configuration of the GridDB client library. The directory below is created when the installation is completed normally.
Installation directory
/usr/griddb/ # installation directory docs/ # document directory lib/ # library directory The directory below is also created when griddb-xx-newsql package is installed. /usr/griddb-xx-newsql-X.X.X/ # NewSQL interface installation directory lib/ # library directory
In addition, the symbolic link below is created.
Symbolic link
/usr/lib64/libgridstore.so -> /usr/lib64/libgridstore.so.0 /usr/lib64/libgridstore.so.0 -> /usr/lib64/libgridstore.so.0.0.0 /usr/lib64/libgridstore.so.0.0.0 -> /usr/griddb-xx-X.X.X/lib/libgridstore.so.0.0.0 /usr/share/java/gridstore.jar -> /usr/griddb-xx-X.X.X/lib/gridstore-X.X.X.jar The file below is also created when griddb-xx-newsql package is installed. /usr/share/java/gridstore-advanced.jar -> /usr/griddb-xx-newsql-X.X.X/lib/gridstore-advanced-X.X.X.jar /usr/share/java/gridstore-jdbc.jar -> /usr/griddb-xx-newsql-X.X.X/lib/gridstore-jdbc-X.X.X.jar
2.6.4 Setting up a library
When the Java version of the client is used, add the client library to CLASSPATH.
$ export CLASSPATH=${CLASSPATH}:/usr/share/java/gridstore.jar
When the C version is used instead, add the client library to LD_LIBRARY_PATH.
$ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib64
When the python client is used, install the python package.
$ pip install /usr/griddb/lib/python
[Memo]
-
In GridDB Advanced Edition, in case of manipulating partition table (container) via Java client, add both a compatibility library and a client library to CLASSPATH. The compatibility library is included in griddb-xx-newsql package.
$ export CLASSPATH=${CLASSPATH}:/usr/share/java/gridstore.jar:/usr/share/java/gridstore-advanced.jar
2.6.5 Configuring client settings
There is no definition file for configuring client settings. Specify the connection point and user/password in the client program.
See “GridDB API reference” (GridDB_API_Reference.html) for specification details on the GridDB NoSQL interface, and “GridDB Advanced Edition JDBC driver instructions” (GridDB_AE_JDBC_DriverGuide.html) for specification details on the GridDB NewSQL interface.
2.7 Uninstallation
Uninstall all packages if GridDB is no longer required. Follow the procedure below to uninstall the packages.
$ su # rpm -e griddb-xx-server # rpm -e griddb-xx-client # rpm -e griddb-xx-java_lib # rpm -e griddb-xx-python_lib # rpm -e griddb-xx-c_lib # rpm -e griddb-xx-docs # rpm -e griddb-xx-newsql
*: xx is the GridDB edition (se, ae, ve)
Files such as definition files and data files etc. that are stored under the GridDB home directory will not be uninstalled.
Manually delete them if they are no longer required.
3 Operations
GridDB operating procedures are explained in this chapter.
The chapter explains the following cases in sequence.
- Operations from start to stop
- Operations when a cluster is in operation
- Troubleshooting
Use the following commands during operations.
[Command list]
Command | Function |
---|---|
gs_startnode | Starting a node |
gs_stopnode | Stop a node (shutdown) |
gs_joincluster | Creating a cluster/joining a node |
gs_leavecluster | Detach a node from a cluster |
gs_stopcluster | Stopping a cluster (stopping a process) |
gs_config | Get cluster configuration node data |
gs_stat | Get node internal data |
gs_appendcluster | Expand a cluster |
gs_failovercluster | Perform the manual failover of a cluster |
gs_partition | Get partition information |
gs_increasecluster | Increase the number of nodes in a cluster |
gs_loadbalance | Enable or disable the autonomous data redistribution in a cluster |
gs_logs | Get node event log |
gs_logconf | Change node log output level |
gs_backup | Backup node data |
gs_backuplist | Check backup data |
gs_restore | Restore backup data |
gs_import | Import data |
gs_export | Export data |
gs_paramconf | Change the memory used by a service |
[Points to note]
- Execute the operating command as a gsadm user.
- If a proxy variable (http_proxy) has been set up, set the address (group) in no_proxy and exclude it from the proxy. As an operating command will perform REST/http communications, the proxy server may be connected by mistake, thereby deactivating the operating command.
- For commands which has the optional setting “connection server: port”, there is no need to specify the option if the port setting has not been changed from the default. In addition, if the optional setting “connection server: port” is specified, this command can be executed from a machine that is different from the one that started the node.
Each procedure of operation controls is explained below. See “GridDB operation control guide” (GridDB_OperationGuide.html) for the details of each command.
3.1 Operations from start to stop
3.1.1 Basic flow
The normal operating flow from commencement until termination of the GridDB cluster (post installation and setup of GridDB nodes) is as follows.
- Start each node.
- Configure the cluster.
- Use the GridDB cluster.
- Stop a cluster.
- Stop each node.
[Points to note]
- For the procedure below, the operation administrator should be aware of the list of host names (or addresses) of all the machines running a node.
- Similarly, the total number of nodes in the cluster should also be known.
- User “admin” and password “admin” are used as examples in the user authentication option (-u).
3.1.2 Starting each node
Execute the start node command on the machine running the node. This command needs to be executed for each node.
However, if the GridDB node process (gsserver) is started automatically by the service, the following start operation is not necessary. Start from the “Composing a cluster” step described in the next section.
Use the following command to start a node.
- gs_startnode
Use the node definition file, cluster definition file and user definition file settings under the conf director of GridDB home directory file to start the node. A command execution example is shown below.
[Command execution example]
$ gs_startnode -u admin/admin -w
The nodes in all the machines that make up the cluster need to be started.
[Points to note]
- When composing a cluster, the cluster definition file of every participant nodes needs to be the same. Prior to starting up the cluster, ensure that every node has the same cluster definition file.
- Similarly, the user definition file of each node also needs to be made the same.
3.1.3 Composing a cluster
Prior to configuring the cluster, start the nodes first. This step is also necessary for a cluster with only a single node. (The cluster is not made up of multiple nodes.)
Execute the following command to add a node into the cluster.
- gs_joincluster [-s connection server: port] -n|--nodeNum number of nodes constituting a cluster -c|--clusterName cluster name -u user name/password
Provide the “cluster name” and “Number of nodes constituting a cluster” as options and execute the command.
Specify the number of nodes in the cluster (“Number of nodes constituting a cluster”) as an option. When starting GridDB for the first time, the number is used as a threshold value for the various services to be started.
An example of the command execution is shown below. In this command, a cluster consisted of “1” node with the name “configured cluster name” is created.
[Command execution example]
$ gs_joincluster -c configured cluster name -n 1 -u admin/admin
An example of the command execution to be performed on a machine that is different from the one in which the nodes have been started is shown below. Another scenario where “3” nodes are added into the “example_three_nodes_cluster” cluster, where the address of the machine is “192.168.10.11”, is shown below.
[Command execution example]
$ gs_joincluster -s 192.168.10.11:10040 -c example_three_nodes_cluster -n 3 -u admin/admin
A cluster will be configured when the correct cluster name is specified and executed for each of the 3 machines making up the cluster. In addition, the cluster service will start only when the number of nodes added into the cluster is equal to the number of nodes specified for the cluster. Once service is started, you will be able to access the cluster from the application.
However, this command will return control immediately after receiving a request. Since the connection from the application could fail before the cluster is fully configured, add “-w” at to the end of the command and wait until the “The GridDB cluster has been started.” message appears, as shown in the example below.
[Command execution example]
$ gs_joincluster -s 192.168.10.12:10040 -c example_three_nodes_cluster -n 3 -u admin/admin $ gs_joincluster -s 192.168.10.13:10040 -c example_three_nodes_cluster -n 3 -u admin/admin -w ... The GridDB cluster has been started.
[Points to note]
- State “1” for the no. of nodes constituting a cluster when a single node configuration is used.
- If the cluster participation command ends in an error, it means that there is a discrepancy in the cluster definition file of the node. Check the cluster definition file again and adopt the same definition.
- The cluster service will not start when the number of nodes participating in a cluster does not reach the number of nodes constituting the cluster. When service is not started, check whether the number of nodes is correct.
Separate the nodes from the cluster if a wrong number of nodes constituting a cluster is specified. Execute the following cluster detachment command.
- gs_leavecluster [-s connection server: port] -u user name/password
An example of the command execution in a machine in which the nodes to be separated from the cluster have been started is shown below.
[Command execution example]
$ gs_leavecluster -u admin/admin
[Points to note]
- If this command is used for the purpose of stopping the cluster, there is a possibility that the data may no longer be viewable after the cluster comes into operation again.
- If the cluster is already in operation, use the cluster stop command (gs_stopcluster).
3.1.4 Using a cluster
After a cluster is composed, data registration and search can be carried out in GridDB from a client program using a registered user.
See the “GridDB API Reference” (GridDB_API_Reference.html) and “GridDB Programming Tutorial” (GridDB_ProgrammingTutorial.html) for details on the creation of client programs.
3.1.5 Changing the memory used by a cluster
The memory used by GridDB is defined in the node definition file of the node that constitutes GridDB. This value can be changed online without having to restart the node or cluster.
Node definition file
Parameters | Data type | Meaning |
---|---|---|
/dataStore/storeMemoryLimit | string | Available memory size |
Execute the following command.
- gs_paramconf [-s connection server: port] -u user name/password --show storeMemoryLimit | --set storeMemoryLimit value
An example to execute a command in a machine in which the nodes have been started is shown below.
[Command execution example]
$ gs_paramconf -u admin/admin --set storeMemoryLimit 2048MB $ gs_paramconf -u admin/admin --show storeMemoryLimit "2048MB"
[Points to note]
- This operation is carried out on a single node unit. If you want to carry out the same changes for all the nodes, perform the operation above for each node.
- When a node is shutdown, changed settings will not be saved. To perpetuate the value, change the node definition file.
3.1.6 Stopping a cluster
Stop a GridDB cluster. To stop the nodes, each node has to be stopped in sequence after the GridDB cluster management process is stopped.
First, stop the cluster management by executing the following stop-cluster command in one of the nodes in the cluster. Execute the following command in one of the nodes participating in the cluster.
- gs_stopcluster [-s connection server: port] -u user name/password
The command line example is shown below.
[Command execution example]
$ gs_stopcluster -u admin/admin
At this point, data registration and search services in all nodes in the cluster will be stopped.
To stop or shut down the nodes, execute the following command. For this purpose, execute a node stop command.
- gs_stopnode [-w [WAIT_TIME]][-s connection server: port] [-f|--force] -u user name/password
The command line example is shown below.
[Command execution example]
$ gs_stopnode -w -u admin/admin
During nodes termination, it may take a while for the node process to be fully terminated due to the checkpoint (file writing of the memory data) process. It is recommended to add the “-w” command to wait until the process is fully completed.
3.1.7 Restarting a stopped cluster
To restart the GridDB cluster, follow the normal startup procedure below.
- Make a note of the number of nodes in the cluster prior to shutting it down.
- Start the nodes.
- Re-add the specified number of nodes to the cluster.
The command line example is shown below.
[Command execution example]
$ gs_startnode -u admin/admin -w ... $ gs_joincluster -c configured cluster name -n 1 -u admin/admin ...
- Use the same cluster name as stated in the cluster definition file.
- For a single node configuration, write “1” for the number of nodes in the cluster, whereas in the case of multiple nodes configuration, use the same number of nodes used prior to the shutdown.
- Information about the number of nodes participating in the cluster is written to the event log file during shutdown.
When restarted, the GridDB will read the database files (transaction log file, checkpoint file) and restore to the original state at the shutdown time. The service will be restarted only once all the “number of nodes constituting a cluster” has been added back to the cluster.
[Points to note]
- The number of nodes at the shutdown point needs to be correctly specified in the “number of nodes constituting a cluster”. The cluster service will not start when the number of nodes added to the cluster is not the same as the number of nodes constituting the cluster. When the service does not start, please check that the correct number of nodes has been used.
- If incorrect “Number of nodes constituting a cluster” was specified, separate the nodes from the cluster using the cluster detachment command while the cluster is not in operation. Ensure that the correct “Number of nodes constituting a cluster” is specified this time prior to re-adding the nodes back into the cluster.
- In addition, when the incorrect number of nodes is specified, there is a possibility that the service starts in the wrong state when the cluster starts operating. In this case, stop and restart the cluster.
- If the current number of nodes becomes less than the original (at the shutdown point), perhaps due to machine failures, etc., specify the number of nodes that can be used and restart the process. Data will be re-arranged as if a failure had occurred during the operation. However, if the nodes get reduced drastically, data may no longer be accessible.
- When adding a new node, add it after the original cluster has been restarted. Please see the “Adding a node to a cluster in operation” section for details.
- It is possible to change the IP address and port (/xxx/serviceAddress, /xxx/servicePort of the node definition file) of the machine when restarting the cluster.
3.2 Getting various data
3.2.1 Getting cluster configuration data
To get the cluster configuration data (data containing the list of nodes participating in a cluster), execute the following command.
- gs_config [-s connection server: port] -u user name/password
An example of the command execution is shown below.
[Command execution example]
$ gs_config -u admin/admin { "follower": [], "master": { "address": "192.168.1.10", "port": 10040 }, "multicast": { "address": "239.0.0.1", "port": 31999 }, "self": { "address": "192.168.1.10", "port": 10040, "status": "ACTIVE" } }
- A list of nodes (address and port) in the cluster excluding the master node will be displayed in “follower”. There may be several units available. This data is displayed in the master node only.
- The address and port of the master node, which manages the cluster, will be displayed in “master”. Make sure it is 1 node only.
- The multicast address and port of the cluster will be displayed in “multicast”.
- The address and port of the node itself which executed the command will be displayed in “self”.
The meaning of system status (Status) is as follows.
- INACTIVE : Not in operation
- ACTIVATING : Start operation
- ACTIVE : In operation
- DEACTIVATING : Start stop
- ABNORMAL : Abnormal stop
- NORMAL_SHUTDOWN : Start normal shutdown
3.2.2 Getting cluster data
To get the cluster data (cluster configuration data and internal data), execute the following.
- gs_stat [-s connection server: port] -u user name/password
An example of the command execution is shown below.
[Command execution example]
$ gs_stat -u admin/admin { : : "cluster": { "activeCount": 3, "clusterName": "defaultCluster", "clusterStatus": "MASTER", : : }
The meaning of the cluster status (clusterStatus) is as follows.
- MASTER : Master
- SUB_MASTER : Candidate serving as the master during a master failure
- FOLLOWER : Follower
- SUB_FOLLOWER : Candidate serving as the follower during a master failure
- SUB_CLUSTER : Cluster is not in operation
The meaning of system status (nodeStatus) is as follows.
- INACTIVE : Not in operation
- ACTIVATING : Start operation
- ACTIVE : In operation
- DEACTIVATING : Start stop
- ABNORMAL : Abnormal stop
- NORMAL_SHUTDOWN : Start normal shutdown
See Parameter list for the description of the other items.
3.2.3 Displaying the event log
To get the most recent event log, use the command below.
- gs_logs [-s connection server: port] -u user name/password --lines No. of rows to acquire [first key word [second key word]]
An example of the command execution is shown below.
[Command execution example]
$ gs_logs -u admin/admin --lines 3 WARNING 2015-02-23T05:28:47.780+0900 host1 728 WARNING EVENT_ENGINE [130901:EE_WAIT_COMPLETION] (queueingElapsed=0, handlerElapsed=10000, watcherEngine=CHECKPOINT_SERVICE, watchedEngine=TRANSACTION_SERVICE, eventType=3004) 2015-02-23T05:29:12.437+0900 host1 726 WARNING IO_MONITOR [1900:CM_LONG_IO] [LONG I/O] sync time,34656,fileName,data/gs_log_0_60.log 2015-02-23T05:29:12.438+0900 host1 726 WARNING IO_MONITOR [LONG EVENT] eventType=PARTITION_GROUP_END, pId=0, pgId=0, time=34658
Event log contains a list of character strings on event data. The format of event data is as follows.
- Time, host name, thread no., event level, generation module, event no., event name, message
Check with the support desk for details.
3.2.4 Displaying and changing the event log output level
Use the command below to display a list of the output levels of the event log.
- gs_logconf [-s connection server: port] -u user name/password
An example of the command execution is shown below.
[Command execution example]
$ gs_logconf -u admin/admin { "levels": { "CHECKPOINT_SERVICE": "INFO", "CHECKPOINT_SERVICE_DETAIL": "ERROR", "CHUNK_MANAGER": "ERROR", "CLUSTER_OPERATION": "INFO", : : } }
Use the command below to change the output level of the event log.
- gs_logconf [-s connection server: port] -u user name/password category output level
An example of the command execution is shown below.
[Command execution example]
$ gs_logconf -u admin/admin CHUNK_MANAGER INFO $ gs_logconf -u admin/admin { "levels": { "CHECKPOINT_SERVICE": "INFO", "CHECKPOINT_SERVICE_DETAIL": "ERROR", "CHUNK_MANAGER": "INFO", "CLUSTER_OPERATION": "INFO", : : } }
A list of the output levels from high to low is shown below.
- ERROR
- WARNING
- INFO
- DEBUG
When the output level is set to low, all logs with an output level higher than that level will be produced. For example, if the level is set at INFO, the INFO, WARNING and ERROR logs will be created.
[Points to note]
- When a node is shutdown, changed settings will not be saved.
- The log output level is either the default value given in gs_node.json of the sample, or a level lower than that is recommended to be set. In addition, the default value is given in Annex Parameter list.
3.3 Backup and restoration
3.3.1 Data backup
In GridDB, a hot backup of node units can be carried out.
Data of a node in operation can be backed up by executing the command below.
- gs_backup [-s connection server: port] -u user name/password backup name
An example of the command execution is shown below.
[Command execution example]
$ cat /var/lib/gridstore/conf/gs_node.json # configuration check { "dataStore":{ "dbPath":"/var/lib/gridstore/data", "backupPath":"/var/lib/gridstore/backup", # backup directory "storeMemoryLimit":"1024MB", "storeWarmStart":false, "concurrency":4, "logWriteMode":1, "persistencyMode":"NORMAL", : : } $ gs_backup -u admin/admin 20130301_backup # backup execution ...
As a result, the following process will be executed.
- Create a 20130301_backup directory under the backup directory (/var/lib/gridstore/backup).
- Create a checkpoint file (gs_cp_n_p.dat), (single or multiple) transaction log file (gs_log_n_m.log), and a backup data file (gs_backup_info.json,gs_backup_info_digest.json) (hereinafter known as a backup file group).
Control returns after commencing backup. Depending on the data size and online processing load, it may several hours or more for the backup to be completed.
The progress status of the backup can be acquired with a gs_stat command.
The progress status of the backup can be backed up by executing the command below.
- gs_stat -t backup [-s connection server: port] -u user name/password
[Command execution example]
$ gs_stat -t backup -u admin/admin 20130301_backup BackupStatus: Processing # backup in progress $ gs_stat -t backup -u admin/admin 20130301_backup BackupStatus: - # backup completed or not in operation
[Points to note]
- See “GridDB backup guide” (GridDB_BackupGuide.html) for the backup details.
- To perform a hot backup of the whole cluster while service is being continued, the above-mentioned backup operation needs to be carried out for all nodes constituting the cluster.
-
In this example, the backupPath is
/var/lib/gridstore/backup
as a matter of convenience but in an actual operation, change the directory to a suitable directory to match the system configuration. - When the system is restored using backup data, the status is restored to the status immediately before the completion of the backup.
- If a failure occurs during backup, the system cannot be restored using this as the backup performed is incomplete.
- When executing a hot backup, the backup may be created with the entire cluster in a non-conforming state if multiple containers have been created. Where necessary, ban transaction services so that the backup can be executed in the static state.
- In GridDB, data will be automatically re-arranged when a failure occurs. As a result, when a failure occurs during a backup, it may no longer be possible to back up the necessary data. When a failure occurs, perform the backup again starting with the first node.
3.3.2 Restoring backup data
To restore backup data to a node, follow the procedure below (this will restore an entire cluster from backup data).
-
Check that no node has been started.
- Check that the cluster definition file is the same as the other nodes in the cluster that the node is joining.
-
Check whether past transaction log files and checkpoint files have been left behind in the database file directory (
/var/lib/gridstore/data
by default) of the node.- Delete if unnecessary and move to another directory if required.
- Execute the restore command on the machine executing the node.
- Start a node.
- Join a cluster.
Execute the following command.
- gs_backuplist -u user name/password
Below is a specific example to display a list of the backup names. A list of the backup names can be displayed regardless of the startup status of the nodes. The Status appears as Processing if the backup process is in progress with the nodes started.
[Command execution example]
$ gs_backuplist -u admin/admin BackupName Status StartTime EndTime ------------------------------------------------------------------------ 20141025NO2 P 2014-10-25T06:37:10+0900 - 20141025 NG 2014-10-25T02:13:34+0900 - 20140925 OK 2014-09-25T05:30:02+0900 2014-09-25T05:59:03+0900 20140825 OK 2014-08-25T04:35:02+0900 2014-08-25T04:55:03+0900
The backup status (Status) will be one of the following.
- OK: Normal
- NG: Error
- P: In progress
[Points to note]
- If the status displayed is NG, the backup file may be damaged and so restoration is not possible.
An execution example to restore backup data is shown below. Stop the nodes first prior to executing the restoration command.
[Command execution example]
$ mv ${GS_HOME}/data/*.{dat,log} ${GS_HOME}/temp # move the database file $ gs_restore 20130521_backup # restore
As a result, the following process will be executed.
-
Copy the backup file group from the 20130521_backup directory under the backup directory (
/var/lib/gridstore/backup
) to the data directory (/var/lib/gridstore/data
).
In this example, the backup directory is /var/lib/gridstore/backup
and the data directory is /var/lib/gridstore/data
as a matter of convenience but in an actual operation, change the directory to a suitable directory to match the system configuration. (See Other Parameters)
After restoration is completed, restart the nodes and re-add them to the original cluster.
[Command execution example]
$ gs_startnode -u admin/admin -w ... $ gs_joincluster -c [Configured cluster name] -n 1 -u admin/admin ...
Once restarted, the node will import the database file (backup file group) and then start the services back on. After import ends, the node will start the services.
[Points to note]
- Take note of the no. of partitions and the parameter of the processing parallelism in the cluster definition file. Set the configuration value of the node to restore to be the same as the configuration value of the backup node. The node cannot start correctly if it is not the same.
- If you want to restore the backup state correctly, the backup and restoration tasks need to be carried out for the entire cluster.
- For example, even if some of the nodes are restored, these nodes cannot be returned to the state they were in at the time of the backup. After restoration, it is necessary to let the nodes join the cluster in operation in order to use the data. However, if the data is updated in the cluster after backup, the restored data will be updated by the (updated) cluster data.
- In particular, if the cluster configuration changes from the time the backup was created, there will be no restoration effect. As the data will be autonomously re-arranged if the node is forced to join a cluster, the data will become invalid with a high probability even if restored.
- If data is missing in the backup data file, or if the contents have been revised, the node will not be able to start services.
3.4 Adding/detaching nodes in a cluster
3.4.1 Adding a node to a cluster in operation
Additional nodes can be added to a running cluster having a specified number of nodes (the number is specified in the cluster configuration gs_joincluster).
Follow the procedure below to add additional nodes to a running cluster.
- Ensure that the cluster is running.
- Check the status of the cluster.
-
Start the nodes that you want to add.
- Check the cluster definition file of the node you want to add is the same as that of the other nodes of the cluster which you want to add the node to.
-
Execute the node addition command on the “node to be added”.
- Get the cluster information of the nodes to be added with a gs_stat command, and if the cluster status turns to FOLLOWER, the node will be able to join the cluster.
To increase the number of nodes, execute the following command.
- gs_appendcluster --cluster connection server: port [-s connection server: port] -u user name/password
Specify the server address and port (for the operating command) of “any one of the nodes constituting the cluster where the node is to be added” in the cluster option. A specific example on appending a new node to a cluster is shown below.
Check the status of the cluster to append the node.
[Command execution example]
$ gs_stat -s 192.168.33.29:10040 -u admin/admin { : "cluster":{ // cluster-related "activeCount":5, // number of nodes already participating in a cluster "clusterName":"function_1", // cluster name "clusterStatus":"MASTER", // cluster status "designatedCount":5, // number of nodes constituting a cluster (predetermined number of nodes) :
A cluster can be added if the number of nodes already participating in a cluster (number of nodes currently joined to a cluster) is equal to the number of nodes constituting a cluster. If the number of nodes constituting a cluster > number of nodes already participating in a cluster, use gs_joincluster (join a cluster configuration) to add a node to the cluster.
Execute the following command on the machine to which the node will be added. Specify the server address and port (for operating command) of any of the nodes in the cluster (the node does not have to be a master node).
[Command execution example]
$ gs_startnode -u admin/admin -w $ gs_appendcluster --cluster 192.168.33.29:10040 -u admin/admin
After appending the nodes, the number of nodes constituting a cluster and the number of nodes already participating in a cluster will be changed.
[Command execution example]
$ gs_stat -u admin/admin { : "cluster":{ // cluster-related "activeCount":6, // number of nodes already participating in a cluster "clusterName":"function_1", // cluster name "clusterStatus":"MASTER", // cluster status "designatedCount":6, // number of nodes constituting a cluster (predetermined number of nodes) :
[Points to note]
- Since the number of nodes constituting a cluster is required during cluster restart, make a note of this number using the gs_stat command when cluster expansion is carried out.
- Non-stop expansion of a GridDB cluster (node increase) will be carried out 1 unit at a time.
- In the case of large scale expansion, stop and re-configure the cluster instead.
3.4.2 Detaching a node from a cluster in operation (shrinking a cluster)
In the case where a single node needs to be removed from a running GridDB cluster, follow the procedure below.
- Check that the cluster is running.
- Execute the cluster detachment command on the node which needs to be detached.
Execute the following cluster detachment command.
- gs_leavecluster [-s connection server: port] [-f] -u user name/password
[Command execution example]
$ gs_leavecluster -u admin/admin
[Points to note]
- If there is a possibility of data lost resulted from node detachment, cluster reduction will not be able to be carried out. To force node detachment, use the -f option.
- Continuous node reduction will be carried out 1 node at a time.
- A cluster will be terminated if the number of nodes existing in the cluster is less than half of the number of nodes constituting the cluster. When a large scale node reduction is required, stop and re-configure the cluster with the new number of nodes. However, please note that when a large scale reduction is carried out, the possibility of data lost increases.
3.5 Software update
3.5.1 Updating the software
Follow the procedure below to update the software.
- Stop the cluster
- Stop the node
- Make a backup copy of the definition file, database file and event log file
- Update the software
- Start the node
- Configure the cluster
An example of the command execution in a machine in which the nodes have been started is shown below.
[Command execution example]
$ gs_stopcluster -u admin/admin $ gs_stopnode -u admin/admin $ cp -rp /var/lib/gridstore/data /xxx/shelter # copy just in case $ cp -rp /var/lib/gridstore/log /xxx/shelter # copy just in case $ cp -rp /var/lib/gridstore/conf /xxx/shelter # copy just in case $ su # rpm -Uvh griddb-xx-server-Y.Y.Y-linux.x86_64.rpm # rpm -Uvh griddb-xx-client-Y.Y.Y-linux.x86_64.rpm # rpm -Uvh griddb-xx-docs-Y.Y.Y-linux.x86_64.rpm # exit $ gs_startnode -u admin/admin -w $ gs_joincluster -c configured cluster name -u admin/admin
*xx: Edition of GridDB to update (se, ae, ve)
*Y.Y.Y: Version of GridDB to update
3.6 Response in the event of a failure
See “GridDB Backup Guide” (GridDB_BackupGuide.html) for details on the failure type and troubleshooting methods.
3.6.1 Basic flow
When a failure occurs, the flow of operations carried out by the administrator will be as follows.
- First, check the cluster operating status and identify which node has been withdrawn from the cluster (failed node).
- Get the failed node event log data.
- Analyze the event log data and check the cause of failure.
- Remove the failed node and replace with a new node.
Even if a failure occurs, GridDB will perform a failover in the cluster to continue services. As long as there is a backup node (replica) in operation, failover will be performed repeatedly. An error will result when there are no more backup nodes.
The following action is carried when a failure occurs in GridDB.
- The failed node is automatically detached from the cluster.
- Failover is carried out in the backup node to replace the failed node.
3.6.2 Checking the operating status of a cluster
The node status can be checked using the get node data command. Check the status individually for all nodes.
[Command execution example]
$ gs_stat -u admin/admin { : "cluster": { : "nodeStatus": "ACTIVE", : } # Repeat execution for all nodes
Check the value of /cluster/nodeStatus output. Symptoms of a failed node are as follows.
-
Detaches itself unexpectedly from the cluster
- When the status is INACTIVE, DEACTIVATING, ABNORMAL, NORMAL_SHUTDOWN.
- Does not return results
3.6.3 Getting event log data from the failed node
To identify the cause of node failure, get the most recent event log from the failed node using the command below.
- gs_logs [-s connection server: port] -u user name/password
An example of the command execution is shown below.
[Command execution example]
$ gs_logs -u admin/admin 2015-03-20T10:07:47.219+0900 host01 22962 INFO CHECKPOINT_SERVICE [30902:CP_STATUS] [CP_START] mode=NORMAL_CHECKPOINT, backupPath= : :
Event log contains a list of character strings on event data. The format of event data is as follows.
- Time, machine name, thread no., event type, event category, generation source file name, generation method, generation row number: followed by any character string (*)
(*) If a new line is returned, a space is assigned to the first character of the new line
Although the cause of failure can be identified directly from the event log data, please check with the service desk for detailed analysis.
[Points to note]
- Due to log rotation, multiple event log files are created. However, only data of the current event log file will be displayed.
3.6.4 Replacing a failed node
Remove the machine running the faulty node from the network. Then, install and set up a GridDB node in a new machine, and add to the new cluster. As explained in the “Adding a node to a cluster in operation” section, the cluster will then automatically redistribute the data and services will be started by the newly added node once redistribution is completed.
4 Annexes
X.X.X below indicates the GridDB version.
4.1 Parameter list
The list of parameters in the node definition file and cluster definition file in GridDB is shown below.
4.1.1 Node definition file (gs_node.json)
Parameters | Data type | Meaning | Default |
---|---|---|---|
/dataStore/dbPath | string | Database file directory | "data" |
/dataStore/backupPath | string | Backup file directory | "backup" |
/dataStore/syncTempPath | string | Long-term sync temporary file directory | "sync" |
/dataStore/storeMemoryLimit | string | Memory buffer size | "1024MB" |
/dataStore/storeWarmStart | boolean | Warm start during restart (false: invalid, true: valid) | false |
/dataStore/concurrency | int | Processing parallelism | 4 |
/dataStore/logWriteMode | int | Log write mode | 1 |
/dataStore/persistencyMode | string | Persistency mode | "NORMAL" |
/dataStore/affinityGroupSize | int | Number of affinity groups | 4 |
/dataStore/storeCompressionMode | string | Mode of block data compression | "NO_COMPRESSION" |
/dataStore/autoExpire | boolean | Automatic deletion of expired data (false: invalid, true: valid) | false |
/checkpoint/checkpointInterval | string | Checkpoint execution interval | "60s" |
/checkpoint/checkpointMemoryLimit | string | Checkpoint memory buffer size | "1024MB" |
/checkpoint/useParallelMode | boolean | Checkpoint parallel operation (false: invalid, true: valid) | false |
/cluster/serviceAddress | string | Reception address used for cluster administration | "127.0.0.1" |
/cluster/servicePort | int | Reception port used for cluster administration | 10010 |
/sync/serviceAddress | string | Reception address used in data synchronization | "127.0.0.1" |
/sync/servicePort | int | Reception port used in data synchronization | 10020 |
/system/serviceAddress | string | Connection address of operation command | "127.0.0.1" |
/system/servicePort | int | Connection port of operation command | 10040 |
/system/eventLogPath | string | Output directory of event log file | "log" |
/transaction/serviceAddress | string | Reception address of transaction process | "127.0.0.1" |
/transaction/servicePort | int | Reception port of transaction process | 10001 |
/transaction/connectionLimit | int | No. of connections upper limit value | 5000 |
/trace/default | string | Event log output level | "LEVEL_ERROR" |
/trace/dataStore | string | "LEVEL_ERROR" | |
/trace/collection | string | "LEVEL_ERROR" | |
/trace/timeSeries | string | "LEVEL_ERROR" | |
/trace/chunkManager | string | "LEVEL_ERROR" | |
/trace/objectManager | string | "LEVEL_INFO" | |
/trace/checkpointFile | string | "LEVEL_ERROR" | |
/trace/checkpointService | string | "LEVEL_INFO" | |
/trace/logManager | string | "LEVEL_WARNING" | |
/trace/clusterOperation | string | "LEVEL_INFO" | |
/trace/clusterService | string | "LEVEL_ERROR" | |
/trace/syncService | string | "LEVEL_ERROR" | |
/trace/systemService | string | "LEVEL_INFO" | |
/trace/transactionManager | string | "LEVEL_ERROR" | |
/trace/transactionService | string | "LEVEL_ERROR" | |
/trace/transactionTimeout | string | "LEVEL_WARNING" | |
/trace/sessionTimeout | string | "LEVEL_WARNING" | |
/trace/replicationTimeout | string | "LEVEL_WARNING" | |
/trace/recoveryManager | string | "LEVEL_INFO" | |
/trace/eventEngine | string | "LEVEL_WARNING" | |
/trace/triggerService | string | "LEVEL_ERROR" |
The parameters used in the GridDB Advanced Edition in addition to those mentioned above are as follows.
Parameters | Data type | Meaning | Default |
---|---|---|---|
/sql/serviceAddress | string | Reception address for JDBC/ODBC client connection | "127.0.0.1" |
/sql/servicePort | int | Reception port for JDBC/ODBC client connection | 20001 |
/sql/storeSwapFilePath | string | SQL intermediate store swap file directory | "swap" |
/sql/storeSwapSyncSize | string | SQL intermediate store swap file and cache size | "1024MB" |
/sql/storeMemoryLimit | string | SQL intermediate store memory buffer size | "1024MB" |
/sql/workMemoryLimit | string | SQL work area memory buffer size | "128MB" |
/sql/workCacheMemory | string | SQL work area memory cache size | "128MB" |
/sql/connectionLimit | int | No. of connections upper limit value | 5000 |
/sql/concurrency | int | Processing parallelism | 4 |
/trace/sqlService | string | Event log output level | "LEVEL_WARNING" |
4.1.2 Cluster definition file (gs_cluster.json)
Parameters | Data type | Meaning | Default |
---|---|---|---|
/dataStore/partitionNum | int | No. of partitions | 128 |
/dataStore/storeBlockSize | string | Block size ("64KB", "1MB") | "64KB" |
/cluster/clusterName | string | Cluster name | "" |
/cluster/replicationNum | int | No. of replicas | 2 |
/cluster/notificationAddress | string | Multicast address for cluster administration | "239.0.0.1" |
/cluster/notificationPort | int | Multicast port for cluster administration | 20000 |
/cluster/notificationInterval | string | Interval of multicast for cluster administration | "5s" |
/cluster/heartbeatInterval | string | Heartbeat interval | "5s" |
/cluster/loadbalanceCheckInterval | string | Load balance check interval | "180s" |
/cluster/notificationMember | array | Address list used in the fixed list method | |
/cluster/notificationProvider/url | string | URL of address provider used in the provider method | |
/cluster/notificationProvider/updateInterval | int | Interval to get list from address provider | "5s" |
/sync/timeoutInterval | string | Short-term synchronization timeout time | "30s" |
/transaction/notificationAddress | string | Address for multi-cast distribution to client | "239.0.0.1" |
/transaction/notificationPort | int | Port for multi-cast distribution to client | 31999 |
/transaction/notificationInterval | string | Interval of multicast distribution to client | "5s" |
/transaction/replicationTimeoutInterval | string | Replication timeout time | "10s" |
/transaction/replicationMode | int | Replication method (0: asynchronous, 1: semi-synchronous) | 0 |
/transaction/authenticationTimeoutInterval | string | Authentication timeout time | "5s" |
The parameters used in the GridDB Advanced Edition in addition to those mentioned above are as follows.
Parameters | Data type | Meaning | Default |
---|---|---|---|
/sql/notificationAddress | string | Address for multi-cast distribution to JDBC/ODBC client | "239.0.0.1" |
/sql/notificationPort | int | Multicast port to JDBC/ODBC client | 41999 |
/sql/notificationInterval | string | Interval of multicast distribution to JDBC/ODBC client | "5s" |
4.2 Build/execution method
An example on how to build and execute a program is covered in this section.
Since the sample program is stored inside the ZIP file in the document package, decompress first using the appropriate program.
[For GridDB NoSQL interface]
For Java
- Set the environmental variables
- Copy the sample program to the test directory
- Build
- Execute
$ export CLASSPATH=${CLASSPATH}:/usr/share/java/gridstore.jar $ mkdir test $ cp /usr/griddb/docs/sample/program/Sample1.java test/ $ javac test/Sample1.java $ java test/Sample1 239.0.0.1 31999 clusterName user password
For C language
- Set the environmental variables
- Copy the sample program
- Build
- Execute
$ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib64 $ cp /usr/griddb/docs/sample/program/sample2.c . $ gcc -lgridstore -o sample2 sample2.c $ ./sample2 239.0.0.1 31999 clusterName user password
For Python
- Copy the sample program
- Execute
$ cp /usr/griddb/docs/sample/program/sample1.py . $ python sample1.py 239.0.0.1 31999 clusterName user password
Specify the 5 arguments, namely the interface address between the client and cluster, the port, the cluster name, user and password.
[For GridDB NewSQL interface]
- Set the environmental variables
- Copy the sample program to the test directory
- Build
- Execute
$ export CLASSPATH=${CLASSPATH}:/usr/share/java/gridstore-jdbc.jar $ mkdir test $ cp /usr/griddb/docs/sample/program/SampleJDBC.java test/ $ javac test/SampleJDBC.java $ java test/SampleJDBC 239.0.0.1 41999 clusterName user password
Specify the 5 arguments, namely the address for multicast distribution to JDBC/ODBC clients, the port, the cluster name, user and password.
[Points to note]
- Execution of sample2 needs to be completed.
[Output example]
DB Connection Start Start 2018-03-17T01:14:30.843Z|false|100.0| End DB Connection Close
5 Trademarks
- GridDB is a trademark of Toshiba Corporation.
- Oracle and Java are registered trademarks of Oracle and/or its affiliates.
- Linux is a trademark of Linus Torvalds.
- Red Hat is a registered trademark or trademark of Red Hat, Inc. in the US and other countries.
- Other product names are trademarks or registered trademarks of the respective owners.
Copyright (c) 2017-2018 Toshiba Digital Solutions Corporation