User Tools

Site Tools


Sidebar

Kerlink Wiki Home Page

Home

Setups

General information

Wirnet™ iBTS information

Wirnet™ iFemtoCell information

Wirnet™ iFemtoCell-evolution information

Wirnet™ iStation information

System management

Network management

LoRa Features

KerOS customization

Support and resources



www.kerlink.com

wiki:support:troubleshoot

Troubleshoot the gateway

Useful logs

When configuring the gateway or when something unusual happens, most of the time, to understand the behaviour of the gateway it is advised to consult the logs. Here is the list of the most useful logs:

  • /var/log/messages*: These files are the main logs, they gather the traces of the firmware. They contain the boot traces, the network events, the hardware events, and many other traces.
  • /var/log/lora.log*: Theses files contain the traces of the common packet forwarder (lorad and lorafwd). If you encounter LoRa issues, this is a good way to start. The verbosity of this log can be increased/decreased. Refer to the log management section.
  • /tmp/board_info.json: This file contains information about the hardware of the gateway. It is generated at each boot time.
  • /tmp/calib_loraloc.json (for Wirnet iBTS): This file contains the calibration values specific to this gateway. It is generated at boot time.
  • /tmp/calib_rf.json (for Wirnet iFemtoCell, Wirnet iFemtoCell-evolution and Wirnet iStation): This file contains the calibration values specific to this gateway. It is generated at boot time.
  • /dev/nmea* (Wirnet iBTS and Wirnet iStation): These devices receive NMEA frames (GPS) every seconds. It can be monitored with the cat command.

Useful commands

Since Kerlink gateways use a Unix system, most current commands are available (ifconfig, route, ps, iptables, …).

KerOS specific commands:

  • gsmdiag.py: generates a status of the modem.
  • connmanctl: used to configure/monitor the network manager.
  • opkg list-installed: lists the installed packages.
  • monit status: displays the status of the monitored processes.

Bootcause

The bootcount value is counting the number of reboots since the last power cycle. At each power loss the counter is reset to 0. The maximum value of this counter is 65535.
The bootcount value can be checked in the file /tmp/board_info.json:

# grep bootcount /tmp/board_info.json
        "bootcount": 1,

When a gateway is started/restarted, U-Boot displays the boot cause.
The possible reasons are:

  • POR: Power On Reset (happens when the gateway is powered or after a powerloss).
  • WDOG: Watchdog.
  • SW: Software Reset.
  • HW: Physical reset (RESET button).
  • UNKNOWN: unknown cause.

This information is transmitted to initrd and to applications with the kernel argument « bootcause ».
The information is then provided in the bootcause parameter in the file /proc/cmdline.
Example:

# cat /proc/cmdline 
console=ttymxc0,115200 bootmode=nominal bootcause=SW bootcount=5 bootfail=0

In case you need assistance to troubleshoot your gateway, Kerlink's support team will help you if:

  • The gateway is still under warranty.
  • The gateway has been bought directly from Kerlink.

If the gateway does not satisfy this requirement, please contact sales@kerlink.fr. Our Sales team will make you an offer to get assistance.

Otherwise, contact support@kerlink.fr to get assistance. Our support team will open you an account in Kerlink's ticketing tool.

When contacting the support team, to improve and speed up the resolution of your issue, please provide them:

  • The serial number of your gateway(s):
    • For Wirnet iBTS, the serial number is written on a sticker inside the case with “Wirnet iBTS” written on it.
    • For Wirnet iFemtoCell, the serial number is written on the rear sticker (it is named “Final product ID”).
    • For Wirnet iStation, the serial number is written on the external sticker (it is named “Product ID”).
    • For Wirnet iFemtoCell-evolution, the serial number is written on rear sticker (it is named “Product ID”).
  • A precise description of your problem. “Gateway not working” is not enough to investigate.
  • The list of the actions you took to troubleshoot/investigate your problem.
  • The date of your problem.
  • The version of your firmware.
  • The logs of your gateway (refer to the automatic log gathering section). Without these logs, it is difficult to investigate. If you are able to reproduce the problem, make sure to reproduce it and gather the logs afterwards.
  • Any information that would help us understand the context. For example, if the gateway is on the field or in a lab, if a third party program is used, if the gateway was previously working, etc.

Automatic log gathering

To simplify the log gathering process, two scripts are available. These scripts generate an archive containing many logs and commands results.

Gathering logs from WebUI since release 5.9

Gathering logs from shell since release 4.2

Gathering logs from shell on older releases

Gathering logs from USB (no commands required)

Helium

This section contains some tips to do the first level of analysis (support level 1) on Helium gateways.
It will help you to investigate before escalate to Kerlink support (level 2).

Please note that there is an existing open wiki for Helium users: Kerlink Helium Hotspot Wiki

  • Second step, if necessary, is to follow the table below:
Source of info KPI Value Action
Onboarding dashboard Last seen More than 30m ago Gateway is offline. Ask customer to check power/internet.
Less than 30m ago Gateway is online. Check next KPI.
Block height = 0 Gateway is downloading a snapshot.
= 1 Gateway is loading the downloaded snapshot.
Else KPI is ok, check next KPI.
Block age > 30 mn Gateway is syncing, check again in one hour. If it persists, you can open a ticket to Kerlink Support.
< 30 mn Gateway is synced. Check next KPI.
Helium explorer 24h/14d earnings > 0 HNT Seems OK. Confirm with other Explorer KPIs.
Last activity date < 3h Seems OK. Confirm with other Explorer KPIs.
Is there a constructed challenge in the past 24h? Yes Seems OK. Confirm with other Explorer KPIs.
No When gateway is synced, it should challenge. You can open a ticket to Kerlink Support.
Is there a challenged beaconer in the past 24h? Yes Seems OK. Confirm with other Explorer KPIs.
No Gateway might be relayed. If not, you can open a ticket to Kerlink Support.
Is there a witnessed beacon in the past 8-24h? Value depends on density around miner Seems OK. Confirm with other Explorer KPIs.
No You can open a ticket to Kerlink Support.
Is there a “broadcasted beacon” in the past 5d? Yes Seems OK. Confirm with other Explorer KPIs.
No This is random but it seems odd. You can open a ticket to Kerlink Support.

Gateways connected to a WMC

For gateways connected to a WMC, you can use the Remote Command: Remote Access > Command

Due its cryptographic key and to ensure Kerlink Helium Hotspot security, no SSH access can be given to those gateways.

You can use commands below:

Command Awaited result
opkg list
monit status miner Status = OK
monit status lorafwd_helium Status = OK
/etc/init.d/miner status Check Diff

If the result is different than the awaited result, you can open a ticket to Kerlink Support.

wiki/support/troubleshoot.txt · Last modified: 2023/06/02 15:47 by ehe