Julia Flux, Недостаточно памяти с функцией точности - PullRequest
Новая
       21

Julia Flux, Недостаточно памяти с функцией точности

1 голос
/ 17 марта 2019

Я изо всех сил пытаюсь заставить мою нейронную сеть работать.У меня есть набор данных с изображениями клеток, у которых малярия или нет (https://www.kaggle.com/iarunava/cell-images-for-detecting-malaria).). И мои данные были упорядочены следующим образом: 

X_training a matrix of dimension 30000×2668, type : Array{Float64,2}
Y_training a matrix of dimension 1×2668, type : Array{Float64,2}

то же самое с X_tests и Y_tests

Моя простая нейронная сеть: 

function simple_nn(X_tests, Y_tests, X_training, Y_training)
    input = 100*100*3
    hl1 = 32
    m = Chain(
      Dense(input, 32, relu),
      Dense(32, 2),
      softmax) |> gpu

    loss(x, y) = crossentropy(m(x), y)

    accuracy(x, y) = mean(onecold(m(x)) .== onecold(y))

    dataset = repeated((X_training, Y_training), 2)
    evalcb = () -> @show(loss(X_training, Y_training))
    opt = ADAM(params(m))

    Flux.train!(loss, dataset, opt, cb = throttle(evalcb, 10))

    println("acc X,Y ", accuracy(X_training, Y_training))

    # Error here
    println("acc tX, tY ", accuracy(X_tests, Y_tests))

end

Моя ошибка: 

ERROR: OutOfMemoryError()
...
Stacktrace:
 [1] * at ./boot.jl:396 [inlined]
 [2] _forward at /home/.../.julia/packages/Flux/8XpDt/src/tracker/lib/array.jl:361 [inlined]
 [3] #track#1 at /home/.../.julia/packages/Flux/8XpDt/src/tracker/Tracker.jl:51 [inlined]
 [4] track at /home/.../.julia/packages/Flux/8XpDt/src/tracker/Tracker.jl:51 [inlined]
 [5] * at /home/.../.julia/packages/Flux/8XpDt/src/tracker/lib/array.jl:349 [inlined]
 [6] Dense at /home/.../.julia/packages/Flux/8XpDt/src/layers/basic.jl:82 [inlined]
 [7] Dense at /home/.../.julia/packages/Flux/8XpDt/src/layers/basic.jl:122 [inlined]
 [8] (::Dense{typeof(relu),TrackedArray{…,Array{Float32,2}},TrackedArray{…,Array{Float32,1}}})(::Array{Float64,2}) at /home/.../.julia/packages/Flux/8XpDt/src/layers/basic.jl:125
 [9] applychain(::Tuple{Dense{typeof(relu),TrackedArray{…,Array{Float32,2}},TrackedArray{…,Array{Float32,1}}},Dense{typeof(identity),TrackedArray{…,Array{Float32,2}},TrackedArray{…,Array{Float32,1}}},typeof(softmax)}, ::Array{Float64,2}) at /home/.../.julia/packages/Flux/8XpDt/src/layers/basic.jl:31
 [10] Chain at /home/.../.julia/packages/Flux/8XpDt/src/layers/basic.jl:33 [inlined]
 [11] (::getfield(Main, Symbol("#accuracy#49")){Chain{Tuple{Dense{typeof(relu),TrackedArray{…,Array{Float32,2}},TrackedArray{…,Array{Float32,1}}},Dense{typeof(identity),TrackedArray{…,Array{Float32,2}},TrackedArray{…,Array{Float32,1}}},typeof(softmax)}}})(::Array{Float64,2}, ::Array{Float64,2}) at /home/.../neural_net.jl:108
 [12] simple_nn(::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,2}) at /home/.../neural_net.jl:118
 [13] top-level scope at util.jl:156

Моя матрица слишком большая?

...