Мы пытаемся настроить некоторые параметры в Python на Windows 10.У нас много параметров и широкий диапазон, поэтому мы пытаемся рассчитать как минимум около 1000000 возможностей.Мы используем многопроцессорность.
Когда мы запускаем код, мы иногда получаем некоторые неописуемые ошибки, и среда IDE не работает.В основном это ошибки времени выполнения.Мы должны перезагрузить компьютер, чтобы IDE снова запустилась.
Мы пытаемся обнаружить эту утечку памяти.Мы попробовали pympler, и это пример вывода:
completed process count: 25 / 64
types | # objects | total size
=========================================================== | =========== | ============
<class 'pandas.core.series.Series | 6 | 16.50 MB
<class 'numpy.float64 | 286312 | 8.74 MB
<class 'str | 108625 | 7.95 MB
<class 'list | 8640 | 7.84 MB
<class 'float | 100209 | 2.29 MB
<class 'int | 1879 | 51.38 KB
<class 'dict | 16 | 2.64 KB
<class 'numpy.ndarray | 8 | 816 B
<class 'pandas.core.internals.managers.SingleBlockManager | 6 | 720 B
<class 'tuple | 9 | 560 B
<class 'pandas._libs.internals.BlockPlacement | 6 | 480 B
<class 'pandas.core.internals.blocks.FloatBlock | 5 | 400 B
<class 'slice | 6 | 384 B
<class 'code | 2 | 288 B
function (<lambda>) | 2 | 272 B
completed process count: 26 / 64
types | # objects | total size
=========================================================== | =========== | ============
<class 'pandas.core.series.Series | 6 | 16.50 MB
<class 'numpy.float64 | 286312 | 8.74 MB
<class 'str | 108625 | 7.95 MB
<class 'list | 8640 | 7.84 MB
<class 'float | 100209 | 2.29 MB
<class 'int | 1879 | 51.38 KB
<class 'dict | 16 | 2.64 KB
<class 'numpy.ndarray | 8 | 816 B
<class 'pandas.core.internals.managers.SingleBlockManager | 6 | 720 B
<class 'tuple | 9 | 560 B
<class 'pandas._libs.internals.BlockPlacement | 6 | 480 B
<class 'pandas.core.internals.blocks.FloatBlock | 5 | 400 B
<class 'slice | 6 | 384 B
<class 'code | 2 | 288 B
Кажется, что размеры и количество объектов не меняются во времени.
Есть какие-нибудь подсказки, как обнаружить преступника?
РЕДАКТИРОВАТЬ:
Я нашел файл журнала:
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 268435456 bytes for Failed to commit area from 0x0000000701600000 to 0x0000000711600000 of length 268435456.
# Possible reasons:
# The system is out of physical RAM or swap space
# The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (c:/BuildAgent/work/dd4f663211fab7c/src/hotspot/os/windows/os_windows.cpp:3272), pid=12164, tid=1876
#
# JRE version: (11.0.2+9) (build )
# Java VM: OpenJDK 64-Bit Server VM (11.0.2+9-b159.34, mixed mode, aot, sharing, tiered, compressed oops, g1 gc, windows-amd64)
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
--------------- S U M M A R Y ------------
Command Line: org.jetbrains.git4idea.http.GitAskPassApp Username for 'https://gitlab.com':
Host: Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz, 8 cores, 15G, Windows 10 , 64 bit Build 17134 (10.0.17134.706)
Time: Fri May 3 09:19:13 2019 Turkey Standard Time elapsed time: 0 seconds (0d 0h 0m 0s)
--------------- T H R E A D ---------------
Current thread (0x000002cd5e5dd000): JavaThread "Unknown thread" [_thread_in_vm, id=1876, stack(0x000000d062400000,0x000000d062500000)]
Stack: [0x000000d062400000,0x000000d062500000]
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0x5da82a]
V [jvm.dll+0x70b6c9]
V [jvm.dll+0x70cbe8]
V [jvm.dll+0x70d2b3]
V [jvm.dll+0x240f18]
V [jvm.dll+0x5d7fc4]
V [jvm.dll+0x5cd115]
V [jvm.dll+0x2fa5eb]
V [jvm.dll+0x2fa55a]
V [jvm.dll+0x2fa432]
V [jvm.dll+0x2ff316]
V [jvm.dll+0x347993]
V [jvm.dll+0x348096]
V [jvm.dll+0x347a93]
V [jvm.dll+0x2d5588]
V [jvm.dll+0x2d66c7]
V [jvm.dll+0x6eaac7]
V [jvm.dll+0x6ec22c]
V [jvm.dll+0x3551c9]
V [jvm.dll+0x6d274e]
V [jvm.dll+0x3bd953]
V [jvm.dll+0x3bfcc1]
C [java.exe+0x3763]
C [java.exe+0x17a0c]
C [KERNEL32.DLL+0x14034]
C [ntdll.dll+0x73691]
--------------- P R O C E S S ---------------
Threads class SMR info:
_java_thread_list=0x000002cd5c2518b0, length=0, elements={
}
Java Threads: ( => current thread )
Other Threads:
0x000002cd5e5f3800 GCTaskThread "GC Thread#0" [stack: 0x000000d062500000,0x000000d062600000] [id=7888]
0x000002cd5e653000 ConcurrentGCThread "G1 Main Marker" [stack: 0x000000d062600000,0x000000d062700000] [id=720]
0x000002cd5e654800 ConcurrentGCThread "G1 Conc#0" [stack: 0x000000d062700000,0x000000d062800000] [id=2832]
[error occurred during error reporting (printing all threads), id 0xc0000005, EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffd9d305687]
VM state:not at safepoint (not fully initialized)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x000002cd5e5d97c0] Heap_lock - owner thread: 0x000002cd5e5dd000
Heap address: 0x0000000701600000, size: 4074 MB, Compressed Oops mode: Non-zero based: 0x0000000701600000
Narrow klass base: 0x0000000000000000, Narrow klass shift: 0
GC Heap History (0 events):
No events
Deoptimization events (0 events):
No events
Classes redefined (0 events):
No events
Internal exceptions (0 events):
No events
Events (0 events):
No events
Dynamic libraries:
0x00007ff77b370000 - 0x00007ff77b3b7000 C:\Program Files\JetBrains\PyCharm 2019.1\jre64\bin\java.exe
0x00007ffdd37a0000 - 0x00007ffdd3981000 C:\WINDOWS\SYSTEM32\ntdll.dll
0x00007ffdd3050000 - 0x00007ffdd3101000 C:\WINDOWS\System32\KERNEL32.DLL
0x00007ffdd01d0000 - 0x00007ffdd0443000 C:\WINDOWS\System32\KERNELBASE.dll
0x00007ffdd2c80000 - 0x00007ffdd2e10000 C:\WINDOWS\System32\USER32.dll
0x00007ffdd0450000 - 0x00007ffdd0470000 C:\WINDOWS\System32\win32u.dll
0x00007ffdd2670000 - 0x00007ffdd2698000 C:\WINDOWS\System32\GDI32.dll
0x00007ffdcff50000 - 0x00007ffdd00e2000 C:\WINDOWS\System32\gdi32full.dll
0x00007ffdcfeb0000 - 0x00007ffdcff4f000 C:\WINDOWS\System32\msvcp_win.dll
0x00007ffdcfbc0000 - 0x00007ffdcfcb8000 C:\WINDOWS\System32\ucrtbase.dll
0x00007ffdc01f0000 - 0x00007ffdc0459000 C:\WINDOWS\WinSxS\amd64_microsoft.windows.common-controls_6595b64144ccf1df_6.0.17134.706_none_fb43a2cb30647007\COMCTL32.dll
0x00007ffdd2e10000 - 0x00007ffdd2eae000 C:\WINDOWS\System32\msvcrt.dll
0x00007ffdd2930000 - 0x00007ffdd2c52000 C:\WINDOWS\System32\combase.dll
0x00007ffdd2f20000 - 0x00007ffdd3044000 C:\WINDOWS\System32\RPCRT4.dll
0x00007ffdd0150000 - 0x00007ffdd01ca000 C:\WINDOWS\System32\bcryptPrimitives.dll
0x00007ffdd3110000 - 0x00007ffdd313d000 C:\WINDOWS\System32\IMM32.DLL
0x0000000000400000 - 0x00000000005bb000 C:\WINDOWS\SysWOW64\winselfprotect.dll
0x00007ffdd3480000 - 0x00007ffdd3542000 C:\WINDOWS\System32\oleaut32.dll
0x00007ffdd3220000 - 0x00007ffdd32c1000 C:\WINDOWS\System32\advapi32.dll
0x00007ffdce810000 - 0x00007ffdce81a000 C:\WINDOWS\SYSTEM32\version.dll
0x00007ffdd26b0000 - 0x00007ffdd270b000 C:\WINDOWS\System32\sechost.dll
0x00007ffdce7f0000 - 0x00007ffdce807000 C:\WINDOWS\SYSTEM32\netapi32.dll
0x00007ffdcf1b0000 - 0x00007ffdcf1be000 C:\WINDOWS\SYSTEM32\NETUTILS.DLL
0x00007ffdba980000 - 0x00007ffdba997000 C:\Program Files\JetBrains\PyCharm 2019.1\jre64\bin\vcruntime140.dll
0x00007ffdbf1f0000 - 0x00007ffdbf28d000 C:\Program Files\JetBrains\PyCharm 2019.1\jre64\bin\msvcp140.dll
0x00007ffd9d020000 - 0x00007ffd9dad6000 C:\Program Files\JetBrains\PyCharm 2019.1\jre64\bin\server\jvm.dll
0x00007ffdd26a0000 - 0x00007ffdd26a8000 C:\WINDOWS\System32\PSAPI.DLL
0x00007ffdcb0d0000 - 0x00007ffdcb0d9000 C:\WINDOWS\SYSTEM32\WSOCK32.dll
0x00007ffdd2eb0000 - 0x00007ffdd2f1c000 C:\WINDOWS\System32\WS2_32.dll
0x00007ffdcc900000 - 0x00007ffdcc923000 C:\WINDOWS\SYSTEM32\WINMM.dll
0x00007ffdcc8d0000 - 0x00007ffdcc8fa000 C:\WINDOWS\SYSTEM32\WINMMBASE.dll
0x00007ffdcfb70000 - 0x00007ffdcfbb9000 C:\WINDOWS\System32\cfgmgr32.dll
0x00007ffdcfb50000 - 0x00007ffdcfb61000 C:\WINDOWS\System32\kernel.appcore.dll
0x00007ffdb96e0000 - 0x00007ffdb96f1000 C:\Program Files\JetBrains\PyCharm 2019.1\jre64\bin\verify.dll
0x00007ffdbde40000 - 0x00007ffdbe009000 C:\WINDOWS\SYSTEM32\DBGHELP.DLL
0x00007ffdbc9a0000 - 0x00007ffdbc9c9000 C:\WINDOWS\SYSTEM32\dbgcore.DLL
0x00007ffdb8df0000 - 0x00007ffdb8e19000 C:\Program Files\JetBrains\PyCharm 2019.1\jre64\bin\java.dll
0x00007ffdb8dd0000 - 0x00007ffdb8de7000 C:\Program Files\JetBrains\PyCharm 2019.1\jre64\bin\zip.dll
0x00007ffdc9fa0000 - 0x00007ffdc9fab000 C:\Program Files\JetBrains\PyCharm 2019.1\jre64\bin\jimage.dll
dbghelp: loaded successfully - version: 4.0.5 - missing functions: none
symbol engine: initialized successfully - sym options: 0x614 - pdb path: .;C:\Program Files\JetBrains\PyCharm 2019.1\jre64\bin;C:\WINDOWS\SYSTEM32;C:\WINDOWS\WinSxS\amd64_microsoft.windows.common-controls_6595b64144ccf1df_6.0.17134.706_none_fb43a2cb30647007;C:\WINDOWS\SysWOW64;C:\Program Files\JetBrains\PyCharm 2019.1\jre64\bin\server
VM Arguments:
java_command: org.jetbrains.git4idea.http.GitAskPassApp Username for 'https://gitlab.com':
java_class_path (initial): C:/Program Files/JetBrains/PyCharm 2019.1/plugins/git4idea/lib/git4idea-rt.jar;C:/Program Files/JetBrains/PyCharm 2019.1/lib/xmlrpc-2.0.1.jar;C:/Program Files/JetBrains/PyCharm 2019.1/lib/commons-codec-1.10.jar;C:/Program Files/JetBrains/PyCharm 2019.1/lib/util.jar
Launcher Type: SUN_STANDARD
[Global flags]
intx CICompilerCount = 4 {product} {ergonomic}
uint ConcGCThreads = 2 {product} {ergonomic}
uint G1ConcRefinementThreads = 8 {product} {ergonomic}
size_t G1HeapRegionSize = 1048576 {product} {ergonomic}
uintx GCDrainStackTargetSize = 64 {product} {ergonomic}
size_t InitialHeapSize = 268435456 {product} {ergonomic}
size_t MarkStackSize = 4194304 {product} {ergonomic}
size_t MaxHeapSize = 4271898624 {product} {ergonomic}
size_t MinHeapDeltaBytes = 1048576 {product} {ergonomic}
uintx NonNMethodCodeHeapSize = 5835340 {pd product} {ergonomic}
uintx NonProfiledCodeHeapSize = 122911450 {pd product} {ergonomic}
uintx ProfiledCodeHeapSize = 122911450 {pd product} {ergonomic}
uintx ReservedCodeCacheSize = 251658240 {pd product} {ergonomic}
bool SegmentedCodeCache = true {product} {ergonomic}
bool UseCompressedClassPointers = true {lp64_product} {ergonomic}
bool UseCompressedOops = true {lp64_product} {ergonomic}
bool UseG1GC = true {product} {ergonomic}
bool UseLargePagesIndividualAllocation = false {pd product} {ergonomic}
Logging:
Log output configuration:
#0: stdout all=warning uptime,level,tags
#1: stderr all=off uptime,level,tags
Environment Variables:
JAVA_HOME=C:\Program Files\Java\jdk1.8.0_171
PATH=C:/Program Files/Git/mingw64/libexec/git-core;C:/Program Files/Git/mingw64/libexec/git-core;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Users\ismetb\bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\ProgramData\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\Java\jdk1.8.0_171\bin;C:\Program Files\Calibre2\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\Gpg4win\..\GnuPG\bin;C:\Android;C:\Windows\System32;C:\Program Files (x86)\EasyFrom Inc\;C:\Program Files\Git\cmd;C:\MATRIKS;C:\MATRIKS\System32\;C:\spark\bin;C:\Program Files\TortoiseGit\bin;C:\Users\ismetb\Anaconda3;C:\Users\ismetb\Anaconda3\Scripts;C:\Users\ismetb\AppData\Local\Microsoft\WindowsApps;C:\Program Files\JetBrains\PyCharm 2018.3.1\bin;C:\Program Files\JetBrains\IntelliJ IDEA 2018.3.5\bin;C:\Program Files\JetBrains\PyCharm 2018.3.4\bin;C:\Users\ismetb\AppData\Local\rodeo\app-2.5.2\bin
USERNAME=ismetb
DISPLAY=:0.0
OS=Windows_NT
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
--------------- S Y S T E M ---------------
OS: Windows 10 , 64 bit Build 17134 (10.0.17134.706)
CPU:total 8 (initial active 8) (4 cores per cpu, 2 threads per core) family 6 model 60 stepping 3, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, lzcnt, ht, tsc, tscinvbit, bmi1, bmi2, fma
Memory: 4k page, system-wide physical 16294M (478M free)
TotalPageFile size 45926M (AvailPageFile size 36M)
current process WorkingSet (physical memory assigned to process): 10M, peak: 10M
current process commit charge ("private bytes"): 62M, peak: 318M
vm_info: OpenJDK 64-Bit Server VM (11.0.2+9-b159.34) for windows-amd64 JRE (11.0.2+9-b159.34), built on Mar 26 2019 11:55:29 by "" with MS VC++ 14.0 (VS2015)
END.