Generating Diagnostic Files in AnzoGraph 3.1

This topic provides instructions for using the system management CLI, azgctl, to retrieve diagnostic files for troubleshooting issues with an Graph Lakehouse 3.1 release. The CLI is in the <install_path>/bin directory. If you encounter an error and the database remains running, you take an x-ray to produce a tarball of the appropriate system diagnostic files. If you encounter an error that crashes the database, you generate a "crash dump" that retrieves the crash-related diagnostic files.

• Taking an X-Ray
• Generating a Crash Dump

Taking an X-Ray

Run the following command on the leader server to take an x-ray on a running database. The result is a tarball that includes historical system records from the specified time period. All flags for specifying a time period are optional. If you omit the options, the resulting x-ray will include the last 24 hours of historical system data.

The system manager interprets time specifications using the system’s local time and converts the timestamps to UTC when starting the x-ray.

azgctl -xray /path/name.xray [ -f ] [ -t ] [ -d <num_days> ] [ -h <num_hours> ] [ -m <num_minutes> ]

Option	Description
/path/name	The path on the file system where you want to save the tarball and the name of the tarball. All x-rays must be named with the .xray extension.
-f	The -f <time> (or --from <time>) option can be used to specify the time to start the system data capture, i.e., omit all of the records from before the specified time. Time must be specified in the following format: YYYY-MM-DD[:HH[:MM]]. For example, -f 2024-01-10:15:00 sets the start time to 3:00 p.m. (local system time) on January 10, 2024.
-t	The -t <time> (or --to <time>) option can be used to specify the time to end the system data capture, i.e., omit all of the records after the specified time. Time must be specified in the following format: YYYY-MM-DD[:HH[:MM]]. For example, -t 2024-01-09:19:30 sets the end time to 7:30 p.m. (local system time) on January 9, 2024.
-d <num_days>	The -d <num_days> (or --days <num_days>) option can be used to specify the number of days to include in the x-ray. The value must be a positive integer.

• When combined with -t or -f , the number of days is relative to the from or to value. For example, -f 2024-01-10 -d 2 means two days starting from 1/10/24 (i.e., 1/10/24 – 1/12/24). And -t 2024-01-10 -d 2 is two days before 1/10/24 (i.e., 1/8/24 – 1/10/24).
• When included without -f or -t, the number of days is relative to the current local system time. For example, -d 2 captures the last 2 days of data starting from now().

| | -h <num_hours> | The -h <num_hours> (or --hours <num_hours>) option can be used to specify the number of hours to include in the x-ray. The value must be a positive integer.

• When combined with -t or -f , the number of hours is relative to the from or to value. For example, -f 2024-01-10:12:00 -h 5 means the 5 hours after 12:00 p.m. on 1/10/24. And -t 2024-01-10:12:00 -h 5 means the 5 hours before 12:00 p.m. on 1/10/24.
• When included without -f or -t, the number of hours is relative to the current local system time. For example, -h 3 captures the last 3 hours of data starting from now().

| | -m <num_minutes> | The -m <num_minutes> (or --minutes <num_minutes>) option can be used to specify the number of minutes to include in the x-ray. The value must be a positive integer.

• When combined with -t or -f , the number of minutes is relative to the from or to value. For example, -f 2024-01-10:12:00 -m 30 means the 30 minutes after 12:00 p.m. on 1/10/24. And -t 2024-01-10:12:00 -m 30 means the 30 minutes before 12:00 p.m. on 1/10/24.
• When included without -f or -t, the number of minutes is relative to the current local system time. For example, -m 30 captures the last 30 minutes of data starting from now().

|

Examples

The following example generates an x-ray that includes the last 24 hours of system data. A tarball named 24hr_errors.xray is written to the /tmp directory.

/opt/cambridgesemantics/anzograph/bin/azgctl -xray /tmp/24hr_errors.xray

The example below captures the last 12 hours worth of data. A tarball named last12hours.xray is written to the /tmp directory.

/opt/cambridgesemantics/anzograph/bin/azgctl -xray /tmp/last12hours.xray -h 12

The example below captures the last two days of data from before 5:00 p.m. on 1/18/24. A tarball named 1-16_to_1-18.xray is written to the /opt/shared/xrays directory.

/opt/cambridgesemantics/anzograph/bin/azgctl -xray /opt/shared/xrays/1-16_to_1-18.xray -t 2024-01-18:17:00 -d 2

Generating a Crash Dump

If you encounter an issue that stops the database, Graph Lakehouse automatically generates diagnostic files. Follow the instructions below to retrieve the files after a crash.

The database does not need to be running to collect the crash dump.

1. Run the following command on the leader server to view a list of the available crash dumps.

   azgctl -crashlist

   For example:

   /opt/cambridgesemantics/anzograph/bin/azgctl -crashlist

   The results show a list of available crash dumps by timestamp. For example:

   Crash ID Time

   520460982 2023-12-28 20:30:35 520457655 2023-12-28 19:01:25

2. Run the following command to retrieve the appropriate crash files. This command creates a tarball that includes the diagnostic files:

   azgctl -crashfetch [ crash_id ] /path/name.xray

   Include the crash_id when you want to retrieve a specific crash dump that is listed in the crash list. Omit the crash ID to retrieve the latest files. All crash dump tarballs must include the .xray extension.

Examples

The following command captures the most recent crash files. The tarball is named latest_crash.xray and it is saved to the /tmp directory.

/opt/cambridgesemantics/anzograph/bin/azgctl -crashfetch /tmp/latest_crash.xray

The example below captures the crash dump for ID 520457655:

/opt/cambridgesemantics/anzograph/bin/azgctl -crashfetch 520457655 /tmp/crash_520457655.xray

You can run azgctl -crashtoss to remove all crash dumps from the server.