File system lock issues still occurring?

#1

The issue was brought up in passing by @mklassen yesterday, so wanted to know if this is still an issue that is happening and if anyone was able to pin down the source of the problem?

My recollection is it was a filesystem issue, so if that is the case, I am wondering if we can solve it by serving the files differently (I recall we were thinking about a dedicated Linux fileserver as an addon), or by making a scratch filespace for active computing, with a filesystem there that is more appropriate for those tasks.

We have some funds potentially for an add on to the server ($80k)

#2

There is definitely room for improvement in the filesystem performance. Last time I used the filesystem extensively - to transfer bulk datasets to the system - the random locked files were endemic. About 50% of my scp transfers - from other on site servers - failed outright and 80% had dangling locked files. When I switched to rsync I had more success, but basically every rsync transfer resulted in unexplained locked files which made it impossible to reorganize by moving files/folders around without restarting the VM.

Moving directories was also extremely slow. Trying to move a directory up one level, i.e. still on the same share/filesystem, was actually so slow that I ended up just deleting and transferring the data via rsync again because that was faster.

#3

That seems like unacceptable performance… I’m bringing this up with Chris again to see what options we have…

#4

I’ve been dealing with this off and on with Jörn’s datashare as well, but no one from his group has complained recently.

I can see if I can dig up the message tree Chris and I have already had about this. From what I remember, though, it seems as if Chris can’t always find a ready reason as to why certain files get locked.

#5

if you can pass along the info to me too (what the complaint was, ie what software/tool they were running) I can also try to reproduce it…

#6

Just forwarded you my entire email tree with Chris. There wasn’t any specific software involved. My experience was more like what Martyn is experiencing with having files locked soon after copying/moving them.

I’ve not run across any one other than Esmin who has had files become locked because a pipeline died before completion.