Fixing Jupyter notebook/lab nfs4 errors on the CBS server

Tutorials
, ,
#1

Problem

The CBS server uses nfs4 for the home directory, which has known issues with sqlite, which causes some issues when Jupyter notebooks (and iPython) on the CBS server.

E.g. you will see these errors pop up in your console related to Jupyter & iPython failing to create temporary db files in your home directory, which may also result in momentary freezes:

[W 2022-04-29 11:23:39.565 ServerApp] The signatures database cannot be opened; maybe it is corrupted or encrypted. You may need to rerun your notebooks to ensure that they are trusted to run Javascript. The old signatures database has been renamed to /home/ROBARTS/alik/.local/share/jupyter/nbsignatures.db.bak and a new one has been created.
[W 2022-04-29 11:23:39.584 ServerApp] Failed commiting signatures database to disk. You may need to move the database file to a non-networked file system, using config option `NotebookNotary.db_file`. Using in-memory signatures database for the remainder of this session.

and

[IPKernelApp] ERROR | Failed to open SQLite history /home/ROBARTS/alik/.ipython/profile_default/history.sqlite (disk I/O error).
[IPKernelApp] ERROR | History file was moved to /home/ROBARTS/alik/.ipython/profile_default/history-corrupt.sqlite and a new file created.

Solution

Fortunately this problem can be solved by making a change to your config files:

Edit or create the following files with the following content:

~/.ipython/profile_default/ipython_config.py:

c = get_config() #gets the configuration
c.HistoryManager.hist_file='/tmp/ipython_hist.sqlite' #changes history file writing to tmp folder

For Jupyter Notebook/Lab users, edit the ~/.jupyter/jupyter_notebook_config.py or ~/.jupyter/jupyter_lab_config.py to include:

c.NotebookNotary.data_dir = "/tmp"

This instructs iPython and Jupyter to use /tmp for these files instead.

1 Like
#2

You can also use an in memory database (https://github.com/ipython/ipython/issues/2845)

c = get_config()
c.HistoryManager.hist_file=':memory:'

History won’t persist or be shared between iPython sessions with an in memory database, but it will be faster and /tmp only has slightly longer persistence as it is cleared on every login sessions.

It might (untested) also be possible to use the sqlite nolock parameter (https://www.sqlite.org/c3ref/open.html) to disable sqlite locking and use your home directory, which would allow you to persist the data beyond your current login session. However there is a risk of database corruption when locking is disabled, particularly if you are using multiple iPython sessions simultaneously.

c = get_config()
c.HistoryManager.hist_file= os.environ['HOME'] +'/.ipython/profile_default/history.sqlite?nolock=1'

Similarly the NotebookNotary.db_file can use :memory: and might also be able to use nolock. (https://github.com/jupyter/jupyter/issues/174)

1 Like