Повторите операцию с данными в Scala - PullRequest
Новая
       12

Повторите операцию с данными в Scala

0 голосов
/ 27 октября 2018

У нас есть миллионы записей, и мы выполняем различные операции (оконные функции, арифметические операции и т. Д.).Нам нужно повторить одно и то же вычисление несколько (настроенных) раз, например, от 10 до 100000 раз.

Могу ли я просто сделать это из этого решения Самый элегантный цикл повторения в Scala ?Есть ли какое-нибудь последнее решение, так как этому вопросу уже более 7 лет.Заранее спасибо! 

1 to n foreach { _ => some.code() }

1 Ответ

0 голосов
/ 27 октября 2018

Я уверен, что с последними версиями Scala все улучшилось. Но, если вы действительно обеспокоены производительностью, тогда избегайте более идиоматического и функционально выглядящего подхода, такого как 1 to n foreach { }, и придерживайтесь большего количества Java, таких как while loop. 

var i = 0
while (i < n) {
  i = i + 1
  some.code()
}

Позвольте мне показать вам разницу с небольшим тестом, 

import org.scalameter._

def sol1(n: Int): Int = {
  var i = 0
  (1 to n).foreach(_ => i = i + 1)
  i
}

def sol2(n: Int): Int = {
  var i = 0
  var j = 0
  while (j < n) {
    j = j + 1
    i = i + 1
  }
  i
}

val n = 10000000

val time1 = config(
  Key.exec.benchRuns -> 200,
  Key.verbose -> true
) withWarmer {
  new Warmer.Default
} measure {
  sol1(n)
}

val time2 = config(
  Key.exec.benchRuns -> 200,
  Key.verbose -> true
) withWarmer {
  new Warmer.Default
} measure {
  sol2(n)
}

println(s"Total time 1: $time1")
println(s"Total time 2: $time2")

Вывод этой программы: 

Starting warmup.
0. warmup run running time: 16.691378 (covNoGC: NaN, covGC: NaN)
1. warmup run running time: 7.993939 (covNoGC: 0.4983, covGC: 0.4983)
2. warmup run running time: 5.447846 (covNoGC: 0.5869, covGC: 0.5869)
3. warmup run running time: 5.621848 (covNoGC: 0.5926, covGC: 0.5926)
4. warmup run running time: 5.594923 (covNoGC: 0.5835, covGC: 0.5835)
5. warmup run running time: 5.691719 (covNoGC: 0.5666, covGC: 0.5666)
6. warmup run running time: 5.688951 (covNoGC: 0.5490, covGC: 0.5490)
7. warmup run running time: 5.63919 (covNoGC: 0.5328, covGC: 0.5328)
8. warmup run running time: 5.929442 (covNoGC: 0.5129, covGC: 0.5129)
9. warmup run running time: 5.650385 (covNoGC: 0.4985, covGC: 0.4985)
10. warmup run running time: 5.761482 (covNoGC: 0.1263, covGC: 0.1263)
11. warmup run running time: 5.501106 (covNoGC: 0.0236, covGC: 0.0236)
Steady-state detected.
Ending warmup.
measurements: 5.410389 ms, 5.531961 ms, 5.46494 ms, 5.530757 ms, 5.524295 ms, 5.482734 ms, 5.670114 ms, 5.440657 ms, 5.966424 ms, 5.833034 ms, 5.743175 ms, 5.437181 ms, 5.816751 ms, 5.472475 ms, 5.622282 ms, 5.570706 ms, 5.684836 ms, 5.498654 ms, 5.444623 ms, 5.515841 ms, 5.612949 ms, 6.106537 ms, 6.093763 ms, 7.504631 ms, 8.34481 ms, 6.710288 ms, 5.270008 ms, 5.793516 ms, 5.328599 ms, 5.436457 ms, 5.142835 ms, 5.560723 ms, 5.438818 ms, 5.457101 ms, 5.296744 ms, 5.29758 ms, 5.357041 ms, 5.35763 ms, 5.269539 ms, 5.163876 ms, 5.278017 ms, 6.992859 ms, 7.393332 ms, 5.683524 ms, 5.432323 ms, 5.219135 ms, 5.496812 ms, 5.277454 ms, 5.620684 ms, 5.205983 ms, 5.314588 ms, 5.299645 ms, 5.357422 ms, 5.180542 ms, 5.488604 ms, 5.311575 ms, 5.568658 ms, 5.664955 ms, 5.255629 ms, 5.542585 ms, 5.260092 ms, 5.447652 ms, 5.245809 ms, 5.579607 ms, 5.428664 ms, 5.293995 ms, 7.463452 ms, 6.4506 ms, 5.364853 ms, 5.481823 ms, 5.590698 ms, 6.432013 ms, 8.275103 ms, 5.803052 ms, 5.830133 ms, 7.927238 ms, 6.11408 ms, 5.32196 ms, 5.621523 ms, 5.34994 ms, 5.732923 ms, 6.26528 ms, 5.63603 ms, 5.395293 ms, 5.785469 ms, 5.406196 ms, 5.491986 ms, 5.541723 ms, 5.683298 ms, 7.141318 ms, 6.265845 ms, 5.694288 ms, 5.659727 ms, 5.3996 ms, 6.135684 ms, 6.290494 ms, 7.097406 ms, 7.371704 ms, 5.63788 ms, 6.005995 ms, 5.206387 ms, 5.409885 ms, 5.467604 ms, 5.291322 ms, 7.920666 ms, 6.665298 ms, 5.477624 ms, 5.45677 ms, 5.262145 ms, 5.074267 ms, 8.562575 ms, 6.57764 ms, 5.336962 ms, 7.109997 ms, 7.261748 ms, 5.571925 ms, 5.277615 ms, 5.530418 ms, 5.437825 ms, 5.663611 ms, 5.304259 ms, 5.461704 ms, 5.373864 ms, 5.446972 ms, 5.42215 ms, 5.526419 ms, 5.415473 ms, 5.608935 ms, 5.416704 ms, 5.542136 ms, 8.38539 ms, 7.73901 ms, 9.586932 ms, 5.973479 ms, 8.139312 ms, 9.123881 ms, 6.810425 ms, 8.459414 ms, 5.713136 ms, 5.534648 ms, 5.498743 ms, 5.723169 ms, 5.903776 ms, 7.227223 ms, 6.55143 ms, 5.406598 ms, 5.520037 ms, 5.239062 ms, 5.324374 ms, 5.199801 ms, 5.479297 ms, 5.347545 ms, 5.154049 ms, 8.426337 ms, 6.500812 ms, 5.425549 ms, 5.938262 ms, 7.821287 ms, 6.020288 ms, 5.264353 ms, 5.25806 ms, 5.204062 ms, 5.33333 ms, 5.309932 ms, 5.721472 ms, 5.265526 ms, 5.578318 ms, 7.897133 ms, 6.131453 ms, 5.423779 ms, 5.581124 ms, 5.131111 ms, 6.953982 ms, 7.534576 ms, 5.891208 ms, 5.285931 ms, 5.636795 ms, 6.055279 ms, 8.205877 ms, 6.005185 ms, 5.472671 ms, 5.409323 ms, 5.516095 ms, 8.539669 ms, 6.0633 ms, 5.395964 ms, 5.128456 ms, 5.335103 ms, 5.145272 ms, 5.677473 ms, 5.169419 ms, 5.920867 ms, 7.905475 ms, 6.064385 ms, 5.454611 ms, 7.303798 ms, 7.224528 ms, 5.487645 ms, 5.282495 ms, 5.37027 ms
  time1: org.scalameter.Quantity[Double] = 5.9224974849999965 ms

Starting warmup.
0. warmup run running time: 1.950349 (covNoGC: NaN, covGC: NaN)
1. warmup run running time: 0.797119 (covNoGC: 0.5936, covGC: 0.5936)
2. warmup run running time: 0.006142 (covNoGC: 1.0652, covGC: 1.0652)
3. warmup run running time: 0.003548 (covNoGC: 1.3346, covGC: 1.3346)
4. warmup run running time: 0.015025 (covNoGC: 1.5364, covGC: 1.5364)
5. warmup run running time: 0.002625 (covNoGC: 1.7180, covGC: 1.7180)
6. warmup run running time: 0.002549 (covNoGC: 1.8798, covGC: 1.8798)
7. warmup run running time: 0.002215 (covNoGC: 2.0275, covGC: 2.0275)
8. warmup run running time: 0.002261 (covNoGC: 2.1642, covGC: 2.1642)
9. warmup run running time: 0.003065 (covNoGC: 2.2911, covGC: 2.2911)
10. warmup run running time: 0.003933 (covNoGC: 2.9893, covGC: 2.9893)
11. warmup run running time: 0.003236 (covNoGC: 0.8713, covGC: 0.8713)
12. warmup run running time: 0.002826 (covNoGC: 0.9369, covGC: 0.9369)
13. warmup run running time: 0.002794 (covNoGC: 0.9592, covGC: 0.9592)
14. warmup run running time: 0.00218 (covNoGC: 0.1958, covGC: 0.1958)
15. warmup run running time: 0.003077 (covNoGC: 0.1947, covGC: 0.1947)
16. warmup run running time: 0.002991 (covNoGC: 0.1896, covGC: 0.1896)
17. warmup run running time: 0.002077 (covNoGC: 0.1974, covGC: 0.1974)
18. warmup run running time: 0.00192 (covNoGC: 0.2168, covGC: 0.2168)
19. warmup run running time: 0.001893 (covNoGC: 0.2469, covGC: 0.2469)
20. warmup run running time: 0.001934 (covNoGC: 0.2162, covGC: 0.2162)
21. warmup run running time: 0.001935 (covNoGC: 0.2094, covGC: 0.2094)
22. warmup run running time: 0.001908 (covNoGC: 0.2133, covGC: 0.2133)
23. warmup run running time: 0.001867 (covNoGC: 0.2118, covGC: 0.2118)
24. warmup run running time: 0.001926 (covNoGC: 0.2175, covGC: 0.2175)
25. warmup run running time: 0.00188 (covNoGC: 0.1680, covGC: 0.1680)
26. warmup run running time: 0.001974 (covNoGC: 0.0309, covGC: 0.0309)
Steady-state detected.
Ending warmup.
measurements: 0.002037 ms, 8.73E-4 ms, 7.64E-4 ms, 7.57E-4 ms, 7.11E-4 ms, 7.06E-4 ms, 0.017457 ms, 8.43E-4 ms, 5.94E-4 ms, 5.83E-4 ms, 5.56E-4 ms, 5.62E-4 ms, 5.57E-4 ms, 5.44E-4 ms, 5.43E-4 ms, 0.0022 ms, 0.001617 ms, 5.77E-4 ms, 5.59E-4 ms, 5.56E-4 ms, 5.61E-4 ms, 5.54E-4 ms, 5.46E-4 ms, 5.61E-4 ms, 5.52E-4 ms, 5.51E-4 ms, 5.5E-4 ms, 5.53E-4 ms, 5.64E-4 ms, 5.36E-4 ms, 5.47E-4 ms, 5.63E-4 ms, 5.71E-4 ms, 5.41E-4 ms, 5.67E-4 ms, 5.4E-4 ms, 5.46E-4 ms, 5.34E-4 ms, 5.39E-4 ms, 5.5E-4 ms, 5.51E-4 ms, 5.54E-4 ms, 5.49E-4 ms, 5.35E-4 ms, 5.71E-4 ms, 5.66E-4 ms, 5.35E-4 ms, 5.52E-4 ms, 5.47E-4 ms, 5.35E-4 ms, 5.44E-4 ms, 5.51E-4 ms, 5.4E-4 ms, 5.63E-4 ms, 5.42E-4 ms, 5.63E-4 ms, 7.86E-4 ms, 5.8E-4 ms, 5.53E-4 ms, 5.54E-4 ms, 5.43E-4 ms, 5.51E-4 ms, 5.53E-4 ms, 5.57E-4 ms, 5.34E-4 ms, 5.6E-4 ms, 5.68E-4 ms, 5.56E-4 ms, 5.67E-4 ms, 5.62E-4 ms, 5.56E-4 ms, 5.7E-4 ms, 5.73E-4 ms, 5.44E-4 ms, 5.6E-4 ms, 5.6E-4 ms, 5.57E-4 ms, 5.54E-4 ms, 7.51E-4 ms, 7.27E-4 ms, 7.21E-4 ms, 7.53E-4 ms, 7.29E-4 ms, 6.74E-4 ms, 5.72E-4 ms, 5.39E-4 ms, 5.77E-4 ms, 5.67E-4 ms, 5.58E-4 ms, 5.53E-4 ms, 5.54E-4 ms, 5.71E-4 ms, 5.73E-4 ms, 5.62E-4 ms, 5.7E-4 ms, 5.47E-4 ms, 5.42E-4 ms, 5.68E-4 ms, 5.64E-4 ms, 5.78E-4 ms, 0.002652 ms, 5.61E-4 ms, 5.6E-4 ms, 5.53E-4 ms, 5.31E-4 ms, 5.56E-4 ms, 5.51E-4 ms, 5.55E-4 ms, 5.41E-4 ms, 8.12E-4 ms, 7.19E-4 ms, 7.83E-4 ms, 6.0E-4 ms, 7.12E-4 ms, 5.56E-4 ms, 5.57E-4 ms, 7.85E-4 ms, 7.37E-4 ms, 0.00132 ms, 5.62E-4 ms, 5.47E-4 ms, 5.57E-4 ms, 5.59E-4 ms, 5.68E-4 ms, 5.43E-4 ms, 5.52E-4 ms, 5.7E-4 ms, 5.48E-4 ms, 5.8E-4 ms, 5.7E-4 ms, 5.41E-4 ms, 5.6E-4 ms, 5.53E-4 ms, 5.63E-4 ms, 5.68E-4 ms, 5.58E-4 ms, 5.66E-4 ms, 5.53E-4 ms, 5.57E-4 ms, 5.86E-4 ms, 5.61E-4 ms, 5.85E-4 ms, 5.63E-4 ms, 0.001266 ms, 0.001472 ms, 5.45E-4 ms, 5.76E-4 ms, 5.57E-4 ms, 5.85E-4 ms, 5.62E-4 ms, 5.69E-4 ms, 5.52E-4 ms, 5.7E-4 ms, 5.6E-4 ms, 5.46E-4 ms, 5.52E-4 ms, 5.62E-4 ms, 5.6E-4 ms, 5.62E-4 ms, 5.53E-4 ms, 5.58E-4 ms, 5.82E-4 ms, 5.72E-4 ms, 5.72E-4 ms, 5.71E-4 ms, 5.54E-4 ms, 5.74E-4 ms, 5.61E-4 ms, 5.52E-4 ms, 5.54E-4 ms, 5.55E-4 ms, 5.53E-4 ms, 5.55E-4 ms, 5.72E-4 ms, 5.68E-4 ms, 5.48E-4 ms, 5.79E-4 ms, 5.8E-4 ms, 5.54E-4 ms, 5.54E-4 ms, 5.76E-4 ms, 5.65E-4 ms, 5.78E-4 ms, 5.74E-4 ms, 5.8E-4 ms, 5.71E-4 ms, 5.44E-4 ms, 5.45E-4 ms, 5.65E-4 ms, 5.65E-4 ms, 5.56E-4 ms, 5.57E-4 ms, 5.52E-4 ms, 5.69E-4 ms, 5.62E-4 ms, 5.33E-4 ms, 5.55E-4 ms, 5.56E-4 ms, 5.46E-4 ms, 5.42E-4 ms
  time2: org.scalameter.Quantity[Double] = 7.049349999999998E-4 ms

Total time 1: 5.9224974849999965 ms
  res0: Unit = ()
Total time 2: 7.049349999999998E-4 ms
  res1: Unit = ()

Как видите, подход while занял 0.0007 мили-секунды, в то время как более элегантный подход занял 5.9224 мили-секунды.

...