Troubleshooting Nightscout issues

The major categories of Nightscout troubleshooting include:

Connectivity. The rig and Nightscout are good friends. Information is usually two-way so long as the rig has access to the internet (aka, online use). When rigs go “offline”, NS will go stale until internet is again available. If you’re having issues with NS and it’s a brand new setup, you’ll want to double check per the below that URL, API secret, etc. are correct.

Mlab size is too big and you need to clean it. See below for how to check the size of, and compact if needed, your mlab database, which can influence what displays in Nightscout.

Future data. Sometimes entries will get time stamped incorrectly, or the device time zones are off. See below for how to resolve.

Top level troubleshooting for rig-Nightscout issues

Setting up your NS hosting site

You will need to make sure that you have setup you site configuration settings in your NS hosting site (usually that means Heroku) according to the docs. See the Nightscout Setup page for help in setting up your NS site. If you don’t add the OpenAPS-specific settings to your setup, the communications with the rig will not work properly.

mLab maintenance

Your NS data is stored in a place called an mLab database. This mLab database is free so long as you stay below a 500mb data limit. Inevitably, after several months of OpenAPS use, you may fill that free data storage limit. Typically, you won’t be notified of that issue...instead you’ll start to notice sudden problems with your NS site when you haven’t done anything different. Strange symptoms include, but aren’t limited to:

  • OpenAPS and Pump pills not working, but still looping and displaying temp basals (devicestatus collection)
  • BG values going stale and dexcom bridge stopping, which may break looping (entries collection)
  • temp basals no longer rendered, but looping still works (treatments collection)
  • careportal treatments (carbs, boluses) no longer displaying properly (treatments collection)

If you are seeing strange behavior in your previously-working-fine NS site, you’ll want to check your mLab database size. Strange behaviour can include missing bolus or other treatment information. You may also get a 500 error (failed to insert record) message in NightScout when trying to save a treatment, such as logging carbs. To access your mLab database, you will need to click on the mLab integration from within your Heroku dashboard as shown below. Based on which symptoms you’re seeing from the above list, start by checking the size of the referenced collection.

mLab access from Heroku dashboard

From that screen, you will need to click on the HOME link near the top left of the page.

mLab home button

The resulting screen will show you the important information to direct where you need to take action. There are TWO areas where the 500 MB data limit can be an issue. One issue area is the size on disk, which is the virtual space that your database takes up. As data is written onto your database, sometimes it is written inefficiently and virtually “spreads out” to take up more room than it normally would. The other issue area is the size which is the actual data stored in your database. Depending on where your issue is, you may need to compact data (if size on disk is the issue) or cleanout data (if size is the issue).

mLab database sizes

Compact data

If size on disk is close to 500mb, you’ll need to compact your database. To begin the compaction process, click on your database name.

mLab database name

Then click on the Tools tab in the screen that opens. Click on the commands button and then select the repairDatabase from the dropdown menu of available commands. At the bottom of the screen, select the Run Command button.

mLab compact tools

Return to your home screen and you will be able to verify the Size on Disk has decreased.

mLab compact verify

Note about inability to compact data. Several people have reported that they get a timeout error when following the above compacting instructions and the size on disk is not reduced. Try again and this text may appear. “If after issuing this command your Sandbox database still has a Size on Disk (file size) greater than 496 MB, see our FAQ for possible explanations.” If you run into this issue, you can send an email to mLab support at support@mlab.com and they will compact the database for you.

Cleanout data

NOTE: Before you cleanout your data, please check out the option to upload (or “donate”) your data anonymously to the OpenAPS Data Commons project. The OpenAPS Data Commons was created to enable a simple way to share data sets from the community, both with traditional researchers who will create traditional research studies, and with groups or individuals from the community who want to review data as part of their own research projects. So before you delete or cleanout any data from your mLab, consider doing an upload to OpenAPS Data Commons first.

If your mLab database issue is size, then you will need to cleanout some of the historical data collected by your NS site. There are two methods to cleanout space and delete data in your mLab database:

  • mLab direct access
  • Nightscout admin tools

mLab Direct Access

  • Login to your mLab database, as shown above, by clicking on the mLab logo in your Heroku dashboard. Your various collections are shown, similar to the screen shot above. The amount of data each collection is using is listed to the far right of each collection’s line. Do not cleanout your profile collection, that is needed by NS. Additionally, if you use tokens for your NS site, you will want to leave the auth_subjects collection intact. The entries collection are typically BG data from your bridge. The treatments collection are data from your careportal entries either through NS or uploaded via the rig (e.g., carbs, boluses, temp basals, notes of pump events, and temp targets). The devicestatus collection houses basically all the information that is presented in the OpenAPS pill display (determine-basal, BG predictions, IOB, COB, etc). If you use NS reports for your endo appointments, try to time the data cleanout for after an appointment so that you can rebuild a treatment history before the next appointment.
  • Click on a collection’s name to open it.

mLab collection select

  • Click on the button that says “Delete all documents in collection” and then confirm the deletion.

mLab collection select

  • You can confirm that your cleanout has resolved the problem, by checking that your database size is below 500 MB now. Click on the Home link in top left. Then review the size of the database as shown (note: Screenshots are from different databases...they are just to show how to navigate and what results you are looking for. Minor inconsistencies in continuity of the screenshots should be ignored).

mLab collection select

Nightscout Admin Tools

If you go to your Nightscout site’s settings (the three horizontal bars in the upper right of your Nightscout site), you will be able to access your Admin Tools for the site. These include some useful quick mLab actions, includig deleting documents from the devicestatus collection similar to the action we took in the steps above. But, it’s worth noting that if you have errant future data in your Nightscout site that is causing problems (usually during time changes or overseas travel), you can clean-out future entries through this portal, too.

mLab collection select

Future data: all of a sudden, Nightscout is no longer showing treatments (bolus, carbs, finger BGs) on the graph or rendering my basals.

If you suddenly find that Nightscout is not showing treatments (bolus, carbs, finger BGs etc.) on the graph; and/or that your basals are no longer being rendered in the blue basal line; but otherwise, everything looks normal and you are looping properly:

You probably somehow got a future-dated treatment. Sometimes data gets recorded into your NS site that can be date stamped into the future - for example, if your CGM, BG meter, OpenAPS rig, or pump had the wrong time or date set.

These future-data will cause problems in rendering (displaying) data correctly, and can usually cause loop failures as well.

To remove future treatments:

Check your NS Admin Tools section (click the three horizontal bars in the upper right of your Nightscout site, then Admin Tools) to easily identify and clean out your database of future data points (press the “remove treatments in the future” button). If the future treatments were caused by a time mismatch, you’ll need to resolve that first, or the future dated treatments may simply be re-uploaded.

How to delete future-dated treaments

Every once in a while, however, that future data point tool will not work effectively because the future data will actually be stored within the devicestatus collection’s information. If that is the case, you should try cleaning out the devicestatus collection, as described above in the Cleanout Data section.

No data is being displayed, or no Nightscout pills are displayed

If you are using a “test pump” that has not received sufficient data in some time, Nightscout pills will NOT be displayed onscreen. Nightscout may also not work if it hasn’t had CGM data in a while - so if you haven’t been using a CGM and uploading CGM data to Nightscout for the past few days, the site may be empty as well. If this happens, simply use this pump in tandem with a CGM so glucose values are recorded and eventually uploaded to Nightscout. Once sufficient data has been collected (and OpenAPS plugin is enabled and saved) the OpenAPS pills should appear automatically. Medtronic CGM users may also need to do this to get their CGM data flowing into Nightscout after a gap in uploading data.

Dexcom CGM users should make sure they have “share” enabled and have actively shared their data with at least one follower, before data will begin flowing to Nightscout. If you don’t want to share your data with another person, you can just follow yourself.

Nightscout pill info is incorrect

There are three pills (aka, information boxes) that are noteworthy about your NS display, and that people commonly interpret as “incorrect” despite all the warnings/explanations in these docs.

  • IOB pill will normally display the IOB reported by your OpenAPS pill. If your loop is failing or NS communications are down because the rig has gone offline, there’s a good possibility that your IOB pill will be displaying an incorrect IOB based on the careportal’s method of calculating IOB (rather than OpenAPS’s way). You can determine the source of your IOB pill’s information by clicking or hovering on the pill. If the pills says “OpenAPS”, then it’s good to use that data. Additionally, it should report the portion of IOB termed “basal IOB”, which is the IOB from of temp basal adjustments and SMBs, if enabled.
  • COB pill should NOT be included on your heroku settings “ENABLE” line. If you go against this advice, you may experience laggy NS performance and see incorrect COB reported in your COB pill on the NS site. Don’t say you haven’t been warned. Until NS dev team can address these issues, the recommendation stands to NOT include COB in your NS site settings.
  • BASAL pill should NOT be used in your NS site. The information on that pill updates so slowly sometimes, that you may incorrectly jump to assumptions that your rig is behaving differently than it actually is. Instead, use the OpenAPS pill to find current information about your current basal rate...or press the ESC button on your pump in order to directly read the current temp basal. Additionally, the basal rendering (the blue lines of the NS display) can sometimes lag by up to 2-5 minutes, depending on loop activities...so again use the OpenAPS pill or pump if you are interested in the most up-to-date information on temp basals.