Thread creation error: Not enough storage is available to process this command

Author Message
Brent 10/23/2011 06:32 am
I am using Windows 7 (64-bit) on a machine with 24 GB of RAM. I have hundreds of gigs of free disk space.

I never ran into this problem using any of the OE 6 Pro betas. As soon as I upgraded to the final version of 6, OE Pro started to freeze after downloading for a while and Windows 7 would close the program. I would sometimes get a dialog box with "Thread creation error: Not enough storage space is available to process this command."

I can't download any site to completion anymore without this happening.

Example sites:
Oleg Chernavin 10/23/2011 06:33 am
Can you please see if you have Log turned on? It may cause slowdowns and high memory usage.

If this is not the issue, can you tell me how much RAM is used before the errors and how many files get downloaded/their size? Thank you!

Best regards,
Oleg Chernavin
MP Staff
Brent 10/24/2011 12:56 pm
Hi Oleg,

I did have logging on. I turned it off and did a few tests. Out of eight sites that I downloaded, 4 crashed after turning off logging. Turning off logging helped with 4 of the 8 sites.

Site Memory Files Notes
------------------------------------------------------------------------------------------------------------------------ 283 MB 25442 WSOE crashed when starting DL 350 MB 22998 crash 295 MB 14767 crash 130 MB 6216 106 MB 208 110 MB 687 181 MB 7060 crash 101 MB 1363

The Memory column is taken from the graph displayed on the System Monitor tab. The File column is the number of files downloaded -- if OE crashed, I took the numbers from the greyed out OE window. If the download was successful, I took the numbers from the status bar when the download finished.

During the downloading of the first site, the WSOE.exe process crashed early on but it continued to download until it finally crashed.

For each site, I left pretty much everything default. I only unchecked the level limit checkbox and set the server to "Load files only within the starting" and selected the Server radio button. I also added the url to the Included keywords tab.
Oleg Chernavin 10/24/2011 12:59 pm
Please ignore WSOE errors for a while. They do not influence the download.

Regarding the other crashes - please try to run Offline Explorer with the following command-line parameters and see if any of them improves the situation:

oe.exe /NoDB
oe.exe /NoMap
oe.exe /SkipOvrCache

Make sure that Log is turned off. And also install the newest version:

Brent 11/03/2011 04:30 pm
Sorry for the delay in reporting back. I installed OE Pro 6.0.3666 and did the tests. I exited OE between each site download.

oe.exe /NoDB (~76.4 MB to load OE Pro 6)

Site Memory Files Notes
------------------------------------------------------------------------------------------------------------------------------ 211 MB 25560 WSOE crashed when starting DL - full download 330 MB 21650 crash (9412 left in queue) 365 MB 28764 crash (14 left in queue) 122 MB 5307 140 MB 178 147 MB 666 130 MB 8461 successful download 101 MB 1363

oe.exe /NoMap (~79.2 MB to load OE Pro 6)

Site Memory Files Notes
------------------------------------------------------------------------------------------------------------------------------ 218 MB 24628 309 MB 27476 crash (8606 left in queue) 103 MB 27814 successful download 113 MB 5286 116 MB 179 121 MB 665 117 MB 8038 successful download -- no disk in the drive warning. 87 MB 491

oe.exe /SkipOvrCache (~79.2 MB to load OE Pro 6)

Site Memory Files Notes
------------------------------------------------------------------------------------------------------------------------------ 227 MB 27955 WSOE crashed when starting DL - full download 256 MB 16440 crash (11286 left in queue) 316 MB 31271 successful download 122 MB 5228 121 MB 178 107 MB 663 113 MB 8038 successful download -- no disk in the drive warning. 92 MB 491

The Strobist site was the only one that crashed all the tests. The photoquotes site resulted in an error I have never seen before with OE -- it complained that there was no disk in the drive. Hitting Retry once made it go away (very weird).
Oleg Chernavin 11/03/2011 04:31 pm
Thank you very much for the tests and details! I am travelling now and I can't reproduce these tests.

I will work on the strobist site on my return to try to catch the error.

Terry 11/05/2011 06:52 am
I had exactly the same problem. The /NoLog had no effect, but checking the logging window in the bottom, I found that even with /NoLog in the command line, it was still logging. Stopping all logging did stop the crash.

Oleg Chernavin 11/05/2011 06:22 pm
/NoLog command stops writing to the Log file on disk (if it was started before with the /Log=c:\dir\file.log command).

Regarding logging and crashes - I asked Brent to turn off logging and he told above that it was done. Still there is some problem, which I have to find and fix.

Oleg Chernavin 11/22/2011 09:36 am
Working on this now. So far downloaded 433 megabytes and the memory usage is not high at all.

Just a thought - there are lots of files with comments on blogspot sites. Perhaps, you do not need them at all. If so, add the following keywords to the Properties - URL Filters - Filename section - Excluded Filename Keywords list:


Oleg Chernavin 11/22/2011 09:55 am
Yes, memory usage is on minimum on my computer. Downloaded over 27,000 files, 618 MBs. Can you try the latest version: