CFMM DICOM Server Upgrade

isolovey · November 12, 2021, 5:28pm

Hello everyone,

This announcement is for those who use the CFMM DICOM web app (Data Browser) for searching and downloading MR datasets. This web app is being replaced by a new version, which can now be accessed at https://dicom.cfmm.uwo.ca/dcm4chee-arc , or via an updated link on https://dicom.cfmm.uwo.ca.

Documentation on how to query for and download datasets is available here.

The old Data Browser will be available at its current URL for several weeks. We encourage you to switch to the new web app in the meantime.

If you access the CFMM DICOM server using the DICOM Query & Retrieve service (e.g. by using a DICOM client, DcmProxy, etc), the connection details will not change. For more details, see the documentation on ways to access data from the DICOM server.

If you have any questions, bug/outage reports or suggestions, please reply in this thread.

switt4 · December 9, 2021, 3:07pm

Hi @isolovey,

A new study recently added ‘bidsdump’ as an authorized user. I can see the study on the old DICOM browser but not the new DICOM browser.

Do we need to update the instructions for users on how to add ‘bidsdump’?

Thanks,

Suzanne

isolovey · December 9, 2021, 5:33pm

@switt4 can you tell me the project name for which you’re seeing a discrepancy between the old an the new DICOM server?

switt4 · December 9, 2021, 5:48pm

@isolovey

Mohsenzadeh^Mental-Imagery

isolovey · December 9, 2021, 8:56pm

@switt4 Our authentication server was caching group memberships for too long, so the addition of bidsdump to the Mohsenzadeh^Mental-Imagery project group was not taking effect. This is now resolved and bidsdump has access to Mohsenzadeh^Mental-Imagery. Sorry about that!

switt4 · December 9, 2021, 9:42pm

Thanks. Glad it was something relatively simple to fix.

tristankk · December 10, 2021, 6:26pm

Hi @isolovey,

I’m having an issue connecting to the DICOM server using a dcm4che invocation that used to work.

Specifically, I’ve been trying something along the lines of: findscu --bind DEFAULT --connect CFMM@dicom.cfmm.uwo.ca:11112 --accept-timeout 10000 --tls-aes --user tkuehn --user-pass ${pswd} -r StudyDescription

This results in a long Java error while trying to establish the connection, including Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: protocol_version

I’ve tried:

Switching the AE title to CFMM-Public – No change.
Switching the hostname to dicom.cfmm.robarts.ca – No change.
Using --tls1 instead of --tls-aes – This seems to connect, but results in the following error:

18:24:37,452 WARN  - DEFAULT->CFMM-Public(1): i/o exception: java.io.EOFException in State: Sta5 - Awaiting A-ASSOCIATE-AC or A-ASSOCIATE-RJ PDU
18:24:37,453 INFO  - DEFAULT->CFMM-Public(1): close Socket[addr=dicom.cfmm.robarts.ca/129.100.47.235,port=11112,localport=47231]
18:24:37,454 DEBUG - DEFAULT->CFMM-Public(1): enter state: Sta1 - Idle
18:24:37,456 WARN  - DEFAULT->CFMM-Public(1): Failed to write A-ASSOCIATE-RQ, will abort association
findscu: null

Do you if there’s something I’m doing wrong? Thanks!

isolovey · December 13, 2021, 4:21pm

@tristankk :

I can connect to the server using dcm4che's findscu. I’m using version 5.24.1.

Has your setup ever worked on the new version of the DICOM server? If so, when did you notice this?

If you’ve previously connected to the old DICOM server and are switching your code over: I recall having issues with an older version of dcm4che, the library/toolkit that includes findscu.

Can you try to download a later version, e.g. 5.24.1, and set it up analogous to how cfmm2tar sets up the Docker image? Basically install the binaries from the zip file provided by dcm4che and then ensure the Western Sectigo certificate (which is the issuer of the TLS certificate for dicom.cfmm.uwo.ca) is trusted by all included tools? That last part is in install_dcm4che_ubuntu.sh.

tristankk · December 13, 2021, 4:56pm

My setup had never worked on the new system, but upgrading dcm4che/findscu did the trick. Thanks for your help!

tristankk · January 13, 2022, 6:21pm

Another comment on the new DICOM server: I’ve been using the new version of cfmm2tar and find that it has a tendency to time out, which I had never come across before. I started accessing new studies around the same time as the server upgrade, so it’s possibly related to the data rather than the server. I now get errors like the following:

findscu: A-ABORT[source: 0 - service-user, reason: 0]
A-ABORT[source: 0 - service-user, reason: 0]
        at org.dcm4che3.net.Association.abort(Association.java:339)
        at org.dcm4che3.net.Timeout.run(Timeout.java:83)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

This is easy enough to work around, but I thought it might be worth bringing up if there’s a fix I’m not aware of.

switt4 · January 13, 2022, 10:25pm

I am also getting a similar error when using procNewScans to manually invoke the autobids system on Compute Canada.

[switt4@gra-login3 ~]$ procNewScansForce '20210709' bd MacDonald
u=rwx,g=rx,o=
study: MacDonald*
date: 20210709
Retrieving dicoms for 'MacDonald*' on '20210709'...
cfmm2tar -c ~/.uwo_credentials.bd -p 'MacDonald*' -d '20210709'   /scratch/switt4/incoming_8946 | tee -a /home/switt4/autobids-pub/var/log_retrieve_cfmm/2022-01-13.txt
WARNING: While bind mounting '/cvmfs:/cvmfs': destination is already in the mount point list
WARNING: While bind mounting '/project:/project': destination is already in the mount point list
WARNING: While bind mounting '/scratch:/scratch': destination is already in the mount point list
WARNING: While bind mounting '/localscratch:/localscratch': destination is already in the mount point list
2022/01/13 05:18:05-ERROR-Picked up _JAVA_OPTIONS: -Xmx2048m
findscu: A-ASSOCIATE-RJ[result: 1 - rejected-permanent, source: 2 - service-provider (ACSE related function), reason: 1 - no-reason-given]
A-ASSOCIATE-RJ[result: 1 - rejected-permanent, source: 2 - service-provider (ACSE related function), reason: 1 - no-reason-given]
	at org.dcm4che3.net.PDUDecoder.nextPDU(PDUDecoder.java:182)
	at org.dcm4che3.net.Association$2.run(Association.java:571)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

I used to occasionally get this error with the previous DICOM server, but it seems to be occurring more frequently with the new DICOM server. Usually, the server will initiate the download when I resubmit the request, but sometimes I have to resubmit the request several times before the server will initiate the download.

switt4 · January 19, 2022, 12:57am

I got the same error as Tristan did when running procNewScans on graham. Immediately resubmitting the command worked, though. The errors, when they occur, appear to be temporary.

[switt4@gra-login3 ~]$ procNewScans '20220118' bd Culham
study: Culham*
date: 20220118
Retrieving dicoms for 'Culham*' on '20220118'...
cfmm2tar -c ~/.uwo_credentials.bd -p 'Culham*' -d '20220118' -U /home/switt4/autobids-pub/var/downloaded_uid.txt /scratch/switt4/incoming_11681 | tee -a /home/switt4/autobids-pub/var/log_retrieve_cfmm/2022-01-18.txt
WARNING: While bind mounting '/cvmfs:/cvmfs': destination is already in the mount point list
WARNING: While bind mounting '/project:/project': destination is already in the mount point list
WARNING: While bind mounting '/scratch:/scratch': destination is already in the mount point list
WARNING: While bind mounting '/localscratch:/localscratch': destination is already in the mount point list
2022/01/18 07:16:59-ERROR-Picked up _JAVA_OPTIONS: -Xmx2048m
findscu: A-ABORT[source: 0 - service-user, reason: 0]
A-ABORT[source: 0 - service-user, reason: 0]
at org.dcm4che3.net.Association.abort(Association.java:339)
at org.dcm4che3.net.Timeout.run(Timeout.java:83)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)

isolovey · January 19, 2022, 8:09pm

These errors are related to an issue with our authentication server (it intermittently cannot access Western’s Active Directory for its queries). I’ll update this thread when a solution to this issue is in place and when these errors should disappear.

Does this cause procNewScans or autobids to fail in production? If so, it might be useful to implement a retry mechanism as a temporary fix.

switt4 · January 21, 2022, 9:28pm

For procNewScans the errors occur almost immediately after submitting the command, so I don’t think a retry is really necessary. It is easy enough for me to just resubmit the local job on compute canada.

You would have to check with @akhanf about whether either or both of these errors have ever showed up on the autobids side. I don’t have access to the log files for the autobids chron job.