Rev 0.54
- apis-log Specification
- Contents
- 1. Terms and Abbreviations
- 2. Overview
- 3. Software Composition
- 4. Detailed Description of Actions
- 5. Data Written to MongoDB
- 6. Configuration Files
- 7. Logging Functions
- 8. Error Handling
- 9. Security
- 10. Privacy
- 11. Open-Source Software Licenses
- 12. Operating Environment
| Term | Explanation |
|---|---|
| apis-main | Software for energy sharing developed by Sony CSL that enables autonomous decentralized control. (For more information, refer to the apis-main Specification Document.) |
| Vert.x | A framework for implementing load-balancing programs. Using Hazelcast, the event bus is clustered and load-balanced. |
apis-log is software for receiving data from apis-main by multicast via a communication line and storing that information in a database in JSON format.
Figure 2-1
The apis-log software architecture is illustrated in Figure 3-1. apis-log runs on the Vert.x event-driven load distribution platform, which runs on top of the Java Development Kit (JDK) on Linux. (Operation has been confirmed on the OSS software version 11. Refer to the OSS license.)
Figure 3-1
The service provided by apis-log is described below.
- Log Receiver
This service receives data from apis-main by UDP multicast via the communication line and stores that information in a MongoDB database in JSON format.
Figure 3-2
This service receives data from apis-main by UDP multicast via the communication line, extracts the required data from UDP1 packets, and then stores that information in a MongoDB database in JSON format. Data written to the database can be encrypted with SSL. (See section 5.1, “Data Written to Log Receiver MongoDB.)
Figure 4-1
The data that is written to the MongoDB database is described below.
| address | IP Address of the log source |
| unitname | Name of the log source |
| thread | Thread name of the log source < Examples > “vert.x-eventloop-thread-X” “vert.x-worker-thread-X” |
| datetime | Date and time the log was sent |
| loglevel | The log level < E.g.: “SEVERE”, “WARNING”, “INFO”, ”FINE”, “FINEST”> |
| loggername | Name of the program that output the log < Examples > “jp.co.sony.csl.dcoes.apis.main.app.controller.DataAcquisition” “jp.co.sony.csl.dcoes.apis.main.app.gridmaster.DataCollection” “jp.co.sony.csl.dcoes.apis.main.app.gridmaster.main_loop.DealExecution” “jp.co.sony.csl.dcoes.apis.main.app.user.ErrorCollection” “com.hazelcast.partition.InternalPartitionService” |
| message | The log message < Examples > “(HARDWARE:LOCAL:ERROR:E032) dcdc mode : CHARGE ; invalid dcdc.meter.ig value : 0.0 ;should between 1.875 and 2.125” “(FRAMEWORK:LOCAL:ERROR:E023)com.hazelcast.core. OperationTimeoutException: No response for 120000 ms.” |
The various configuration files used by apis-log are described below.
The following basic information for configuring apis-log is set in a JSON-format file. The file is read once when the program starts up, so if a parameter is changed, apis-log must be restarted.
| programId | Program identification string |
| comminityId | Community identification string; higher-level ID when organizing more than one cluster |
| clusterId | Cluster identification string; used as encryption seed, together with comminityId and the apis-main version string |
| logReceiver.ipv6 | A flag that indicates whether IPv6 is enabled or disabled Enabled: True Disabled: False |
log.Receiver.multicast GroupAddress |
An IP Address that specifies a particular group for multicasting |
| log.Receiver.port | Log Receiver port number (port = 8888) When there are two clusters on a communication line, they are given different port numbers specified by log.Receiver.port. |
| log.Receiver.printToStdout | A flag that indicates whether Log Receiver Log Console output is enabled or disabled Enabled: True Disabled: False |
| mongoDbWriter.enabled | A flag that indicates whether writing to MongoDB is enabled or disabled. Enabled: True Disabled: False |
| mongoDbWriter.level | Log level for writing to the MongoDB database (Logs of Log Level or higher are written to the database.) |
| mongoDbWriter.host | The MongoDB IP address |
| mongoDbWriter.port | The MongoDB port number |
| mongoDbWriter.ssl | A flag that indicates whether SSL is enabled or disabled for MongoDB communication. Enabled: True Disabled: False |
| mongoDb.database | The name of the “database” data structure for writing to the MongoDB database. |
| mongoDb.collection | The name of the “collection” data structure for writing to the MongoDB database. |
| watchdog.enabled | A flag that indicates whether Watch Dog is enabled or disabled. |
| watchdog.periodMsec | The watchdog reset period (ms) |
| watchdog.host | The IP address accessed by the watchdog service at the interval specified by periodMsec |
| watchdog.port | The port number accessed by the watchdog service at the interval specified by periodMsec |
| watchdog.uri | The watchdog service URI |
watchdog.request TimeoutMsec |
The timeout time of the watchdog service (ms) |
This file contains settings that are related to log output of the standard java.util.logging API (log file location, log capacity, logging level, etc.).
This shell script is executed automatically to start up apis-log when the OS boots up.
The command in start.sh that starts up apis-log is shown below.
java -Djava.net.preferIPv4Stack=true -Duser.timezone=Asia/Tokyo -Djava.util.logging.config.file=./logging.properties -jar ./apis-log-2.23.0-a01-fat.jar -conf ./config.json -cp ./ -cluster &
The arguments that follow ‘java’ in the command are explained below.
-
Djava.net.preferIPv4Stack=true
-> Option for binding to an IPv4 address -
Duser.timezone=Asia/Tokyo
-> Time zone setting -
Djava.util.logging.config.file=./logging.properties
-> Option for specifying the log configuration file -
jar ./apis-log-2.23.0-a01-fat.jar
-> Option for specifying execution of the program encapsulated in a JAR file -
conf ./config.json
-> Option for specifying the configuration file -
cp ./
-> Option for specifying the location of the cluster.xml file
This file holds a shell script that stops execution of the apis-log process.
After executing the shutdown function (stop) via the event bus, the script forcefully terminates its own Java process when a timeout occurs. The timeout time can be set in units of seconds within the script.
The standard Java java.util.logging API is used for logging. Of the seven available log levels described below, the “CONFIG” and “FINER” levels are not used for APIS. The APIS operation logs are described in the logging.properties file, where the log file location, log levels, maximum log size, and maximum number of logs are set.
[java.util.logging Log Level]
- SEVERE
- This level is used when an execution error occurs
Log output at this level can be considered an indication that a problem has occurred.- < Example >
- Analysis of a log received by UDP fails.
- < Example >
- WARNING
- This level is used to issue a warning when there is unexpected behavior that is not an execution error.
Although this level is available in APIS, it is not used for apis-log.
- INFO
- This level is used to output information during normal execution. For apis-log, it is used when event processing that is particularly important to operation is performed.
- < Example >
- Connection information for MongoDB
- < Example >
- CONFIG
- This level relates to configuration settings, but is not output for apis-log.
- FINE
- This level is used for output of ordinary operation information during normal system operation, but it is not output for apis-log.
- FINER
- This level is for information related to the starting and stopping of specific processes, but is not output for apis-ccc.
- FINEST
- This level is used for output of ordinary operation information during normal system operation.
- < Example >
- When Vert.x Verticle is started up, etc.
- < Example >
The apis-log operation logs are output to three destinations: UDP, console, and file. The log output destination and output level can be set in the logging.properties file. Since UDP is output to the communication line, communication traffic and the risk of data leakage should be considered when setting the UDP. In addition, the capacity of non-volatile storage should be considered when outputting to a file.
When a problem occurs, the log is output to file, UDP, and console, but there is no function for the process to reset or stop itself.
Communication between apis-log and MongoDB is SSL-encrypted, with the public or signed certificate provided by MongoDB.
- Communication between apis-log and MongoDB
- SSL public-key cryptography (RSA)
- SSL self-signed certificate
It is necessary to check whether or not the information that can be obtained via apis-log is personal information, because what is considered to be personal information varies with the region of introduction. Caution is required, because the act of sending personal information to an external server without the permission of the information owner may be subject to personal information protection regulations such as GDPR.
The software used by apis-log and the relevant open-source software licensing information is listed in the table below. Because Adopt OpenJDK is used by apis-log only as a linked library, it is not necessary to open the apis-ccc source code, even under GPL v2 with Classpath Exception applied. For the other software, also, no license requires the apis-log source code to be published.
■ Open-source software licensing of software used by apis-log
| Software | Version | License | Code Change |
|---|---|---|---|
| Adopt OpenJDK | 11.0.4+11 | GPLv2 with Classpath Exception | None |
| Vert.x | 3.7.1 | Dual license (select from the following) Eclipse Public License 2.0 Apache License 2.0 |
None |
| Netty | 4.1.8 | Apache License 2.0 | None |
| FasterXML/Jackson | 2.7.4 | Apache License 2.0 | None |
*Software versions may change for various reasons.
The hardware requirements for apis-log are described below.
| CPU processor | 600 to 1000 MHz, 64 bit single-core, 32 KB L1 cache or more ARMv8 recommended |
| DRAM | DDR3, 1.6 Gbps, 1 GB or more |
| Internal storage | 8 GB or more |
| Ethernet | 20 Mbps, 1 port or more, IPv4 and IPv6 support |
The OS requirements for IoT boards used with apis-log use are described below.
| Type | 64-bit OS, Linux recommended(If an OS other than Linux is used, it is necessary to confirm APIS operation.) |
| Size | Depends on the capacity of the IoT Board (3 GB reserved for storing APIS logs, etc.) |
| Operating software | AdoptOpenJDK (32/64bit) |
| OSS licensing | To avoid the effects of copyleft licenses such as GPL, it is prohibited to use an OS that run software covered by such licenses together with apis-log as a single executable file (RTOS, etc.). |
| Other | It shall be possible to run apis-ccc automatically when the OS boots. |
| ssh login/scp file transfer shall be possible. | |
| Logs can be periodically renamed, compressed, or deleted with logrotage, etc. | |
| A fixed IPv4 address shall be possible. | |
| Time synchronization with an ntp server shall be possible. |
