Topic: Newunit problem(?)

I use the newer method of opening files -- Open(newunit=lfn, etc)

I have a program where I did this but after a certain number of opening files, I got the fatal error "too many open files" even though I closed out each one after processing them.

I fixed it by using assignments for the fiiles:
file1=20  - Open(Unit=file1, etc)

These are processing fine on even longer lists of files.

Linda

-----
Linda

Re: Newunit problem(?)

Linda,

That seems like a surprisingly odd bug.  Thank you for reporting it!  I'll have a look at the runtime library to see why it's failing.

Jeff Armstrong
Approximatrix, LLC

Re: Newunit problem(?)

jeff wrote:

That seems like a surprisingly odd bug.  Thank you for reporting it!  I'll have a look at the runtime library to see why it's failing.

I thought so as well -- seems like from a pool of "available unit numbers" it's not releasing when closed.

If it helps, the list it failed on had 3253 items in it, it failed on #2398 or #2397 in the list.

Linda

-----
Linda

Re: Newunit problem(?)

Linda,

I've looked into the issue, and I think the error isn't as simple as I originally thought.  There is no situation in the runtime library where the library would a) run out of unit numbers less than 2**32 possibilities or thereabouts or b) produce the error message you've reported.  It looks like the actual error is being generated by the operating system. 

We had another user reporting an issue privately where rapid file open/close operations would intermittently fail.  I believe this behavior was because, after a file is closed, Windows doesn't necessarily immediately release a handle to the file.  It might take a fraction of a second.  In your case, I think something very similar is occurring.  When you close a file, Windows is taking its time (a fraction of a second) to release the handle.  When you're opening a large number of files quickly, you could end up hitting the max file limit even though you're closing them.

There are two "not great" solutions.  First, you could sleep in your program every so often to allow Windows to take care of some bookkeeping.  I don't particularly love this solution, but it should work. 

Second, you could increase the Windows runtime library's allowed number of open files.  There is a low-level call, _setmaxstdio, that can increase the number of files allowed to be open by a process, and I think you're hitting that limit.  In Fortran you would need to wrap it:

interface
    function setmaxstdio(x) bind(c, name="_setmaxstdio")
    use iso_c_binding, only: c_int
    integer(kind=c_int), value::x ! The limit to set, probably want it to be 1024 or higher
    integer(kind=c_int)::setmaxstdio ! Returns the new setting
    end function setmaxstdio
end interace

Stream I/O, which I think the Fortran runtime is using, limits the open files to 512.  If the files aren't closing in a timely manner, calling the above at the beginning of your program and setting a much higher limit might fix the issue.

Jeff Armstrong
Approximatrix, LLC

Re: Newunit problem(?)

Thanks for looking into it.  Figures that it would be a Windows problem. 

I've got my work around (which is just a straightforward old way of opening files) for now. But will take your advice under consideration for future.

Linda

-----
Linda