Kyoto Cabinet HashDB inserts are not O(1) - tokyo-cabinet
I am experimenting with Kyoto Cabinet HashDB. My database will grow up to the size of 110 TB. The keys are 20 Bytes and values are 128 KB. I am trying to insert about 500 million records. Here is the best performance I have gotten so far:
Time is in seconds. 'n' is the number of 10000 records entered. For example 1st 10000 records took 1 second. 5th 10000 records took 5 seconds and so on. As you can see the performance starts degrading after about 2 million records are entered. I created the database with bucket number set to 7 Billion, and everything else is default. I am using the C/C++ library, and setting memory map capacity to 30 GB (I have 32 GB RAM) using db.tune_map(30LL << 30);.
Inserted 10000 records. n: 1. Time: 1
Inserted 10000 records. n: 2. Time: 1
Inserted 10000 records. n: 3. Time: 1
Inserted 10000 records. n: 4. Time: 1
Inserted 10000 records. n: 5. Time: 4
Inserted 10000 records. n: 6. Time: 4
Inserted 10000 records. n: 7. Time: 4
Inserted 10000 records. n: 8. Time: 4
Inserted 10000 records. n: 9. Time: 4
Inserted 10000 records. n: 10. Time: 4
Inserted 10000 records. n: 11. Time: 4
Inserted 10000 records. n: 12. Time: 4
Inserted 10000 records. n: 13. Time: 4
Inserted 10000 records. n: 14. Time: 4
Inserted 10000 records. n: 15. Time: 4
Inserted 10000 records. n: 16. Time: 4
Inserted 10000 records. n: 17. Time: 4
Inserted 10000 records. n: 18. Time: 4
Inserted 10000 records. n: 19. Time: 7
Inserted 10000 records. n: 20. Time: 4
Inserted 10000 records. n: 21. Time: 4
Inserted 10000 records. n: 22. Time: 4
Inserted 10000 records. n: 23. Time: 5
Inserted 10000 records. n: 24. Time: 4
Inserted 10000 records. n: 25. Time: 4
Inserted 10000 records. n: 26. Time: 4
Inserted 10000 records. n: 27. Time: 4
Inserted 10000 records. n: 28. Time: 4
Inserted 10000 records. n: 29. Time: 5
Inserted 10000 records. n: 30. Time: 4
Inserted 10000 records. n: 31. Time: 4
Inserted 10000 records. n: 32. Time: 4
Inserted 10000 records. n: 33. Time: 4
Inserted 10000 records. n: 34. Time: 4
Inserted 10000 records. n: 35. Time: 4
Inserted 10000 records. n: 36. Time: 4
Inserted 10000 records. n: 37. Time: 5
Inserted 10000 records. n: 38. Time: 4
Inserted 10000 records. n: 39. Time: 4
Inserted 10000 records. n: 40. Time: 4
Inserted 10000 records. n: 41. Time: 4
Inserted 10000 records. n: 42. Time: 4
Inserted 10000 records. n: 43. Time: 4
Inserted 10000 records. n: 44. Time: 4
Inserted 10000 records. n: 45. Time: 4
Inserted 10000 records. n: 46. Time: 4
Inserted 10000 records. n: 47. Time: 4
Inserted 10000 records. n: 48. Time: 5
Inserted 10000 records. n: 49. Time: 4
Inserted 10000 records. n: 50. Time: 4
Inserted 10000 records. n: 51. Time: 5
Inserted 10000 records. n: 52. Time: 4
Inserted 10000 records. n: 53. Time: 5
Inserted 10000 records. n: 54. Time: 5
Inserted 10000 records. n: 55. Time: 4
Inserted 10000 records. n: 56. Time: 5
Inserted 10000 records. n: 57. Time: 5
Inserted 10000 records. n: 58. Time: 5
Inserted 10000 records. n: 59. Time: 5
Inserted 10000 records. n: 60. Time: 5
Inserted 10000 records. n: 61. Time: 4
Inserted 10000 records. n: 62. Time: 5
Inserted 10000 records. n: 63. Time: 5
Inserted 10000 records. n: 64. Time: 4
Inserted 10000 records. n: 65. Time: 6
Inserted 10000 records. n: 66. Time: 4
Inserted 10000 records. n: 67. Time: 4
Inserted 10000 records. n: 68. Time: 7
Inserted 10000 records. n: 69. Time: 6
Inserted 10000 records. n: 70. Time: 7
Inserted 10000 records. n: 71. Time: 7
Inserted 10000 records. n: 72. Time: 6
Inserted 10000 records. n: 73. Time: 10
Inserted 10000 records. n: 74. Time: 7
Inserted 10000 records. n: 75. Time: 10
Inserted 10000 records. n: 76. Time: 9
Inserted 10000 records. n: 77. Time: 8
Inserted 10000 records. n: 78. Time: 10
Inserted 10000 records. n: 79. Time: 8
Inserted 10000 records. n: 80. Time: 6
Inserted 10000 records. n: 81. Time: 8
Inserted 10000 records. n: 82. Time: 8
Inserted 10000 records. n: 83. Time: 9
Inserted 10000 records. n: 84. Time: 9
Inserted 10000 records. n: 85. Time: 9
Inserted 10000 records. n: 86. Time: 9
Inserted 10000 records. n: 87. Time: 7
Inserted 10000 records. n: 88. Time: 10
Inserted 10000 records. n: 89. Time: 8
Inserted 10000 records. n: 90. Time: 10
Inserted 10000 records. n: 91. Time: 10
Inserted 10000 records. n: 92. Time: 10
Inserted 10000 records. n: 93. Time: 10
Inserted 10000 records. n: 94. Time: 10
Inserted 10000 records. n: 95. Time: 10
Inserted 10000 records. n: 96. Time: 11
Inserted 10000 records. n: 97. Time: 9
Inserted 10000 records. n: 98. Time: 10
Inserted 10000 records. n: 99. Time: 10
Inserted 10000 records. n: 100. Time: 11
Inserted 10000 records. n: 101. Time: 10
Inserted 10000 records. n: 102. Time: 11
Inserted 10000 records. n: 103. Time: 11
Inserted 10000 records. n: 104. Time: 11
Inserted 10000 records. n: 105. Time: 12
Inserted 10000 records. n: 106. Time: 11
Inserted 10000 records. n: 107. Time: 12
Inserted 10000 records. n: 108. Time: 12
Inserted 10000 records. n: 109. Time: 10
Inserted 10000 records. n: 110. Time: 11
Inserted 10000 records. n: 111. Time: 10
Inserted 10000 records. n: 112. Time: 10
Inserted 10000 records. n: 113. Time: 11
Inserted 10000 records. n: 114. Time: 10
Inserted 10000 records. n: 115. Time: 11
Inserted 10000 records. n: 116. Time: 10
Inserted 10000 records. n: 117. Time: 10
Inserted 10000 records. n: 118. Time: 10
Inserted 10000 records. n: 119. Time: 10
Inserted 10000 records. n: 120. Time: 11
Inserted 10000 records. n: 121. Time: 10
Inserted 10000 records. n: 122. Time: 11
Inserted 10000 records. n: 123. Time: 9
Inserted 10000 records. n: 124. Time: 11
Inserted 10000 records. n: 125. Time: 9
Inserted 10000 records. n: 126. Time: 11
Inserted 10000 records. n: 127. Time: 10
Inserted 10000 records. n: 128. Time: 10
Inserted 10000 records. n: 129. Time: 12
Inserted 10000 records. n: 130. Time: 10
Inserted 10000 records. n: 131. Time: 10
Inserted 10000 records. n: 132. Time: 10
Inserted 10000 records. n: 133. Time: 10
Inserted 10000 records. n: 134. Time: 16
Inserted 10000 records. n: 135. Time: 10
Inserted 10000 records. n: 136. Time: 10
Inserted 10000 records. n: 137. Time: 10
Inserted 10000 records. n: 138. Time: 10
Inserted 10000 records. n: 139. Time: 9
Inserted 10000 records. n: 140. Time: 10
Inserted 10000 records. n: 141. Time: 10
Inserted 10000 records. n: 142. Time: 10
Inserted 10000 records. n: 143. Time: 10
Inserted 10000 records. n: 144. Time: 11
Inserted 10000 records. n: 145. Time: 10
Inserted 10000 records. n: 146. Time: 11
Inserted 10000 records. n: 147. Time: 9
Inserted 10000 records. n: 148. Time: 10
Inserted 10000 records. n: 149. Time: 9
Inserted 10000 records. n: 150. Time: 10
Inserted 10000 records. n: 151. Time: 20
Inserted 10000 records. n: 152. Time: 18
Inserted 10000 records. n: 153. Time: 33
Inserted 10000 records. n: 154. Time: 25
Inserted 10000 records. n: 155. Time: 30
Inserted 10000 records. n: 156. Time: 18
Inserted 10000 records. n: 157. Time: 12
Inserted 10000 records. n: 158. Time: 13
Inserted 10000 records. n: 159. Time: 14
Inserted 10000 records. n: 160. Time: 10
Inserted 10000 records. n: 161. Time: 9
Inserted 10000 records. n: 162. Time: 12
Inserted 10000 records. n: 163. Time: 11
Inserted 10000 records. n: 164. Time: 11
Inserted 10000 records. n: 165. Time: 11
Inserted 10000 records. n: 166. Time: 11
Inserted 10000 records. n: 167. Time: 12
Inserted 10000 records. n: 168. Time: 10
Inserted 10000 records. n: 169. Time: 11
Inserted 10000 records. n: 170. Time: 12
Inserted 10000 records. n: 171. Time: 11
Inserted 10000 records. n: 172. Time: 10
Inserted 10000 records. n: 173. Time: 11
Inserted 10000 records. n: 174. Time: 11
Inserted 10000 records. n: 175. Time: 10
Inserted 10000 records. n: 176. Time: 11
Inserted 10000 records. n: 177. Time: 11
Inserted 10000 records. n: 178. Time: 10
Inserted 10000 records. n: 179. Time: 11
Inserted 10000 records. n: 180. Time: 11
Inserted 10000 records. n: 181. Time: 11
Inserted 10000 records. n: 182. Time: 10
Inserted 10000 records. n: 183. Time: 11
Inserted 10000 records. n: 184. Time: 11
Inserted 10000 records. n: 185. Time: 11
Inserted 10000 records. n: 186. Time: 11
Inserted 10000 records. n: 187. Time: 11
Inserted 10000 records. n: 188. Time: 11
Inserted 10000 records. n: 189. Time: 11
Inserted 10000 records. n: 190. Time: 11
Inserted 10000 records. n: 191. Time: 11
Inserted 10000 records. n: 192. Time: 13
Inserted 10000 records. n: 193. Time: 13
Inserted 10000 records. n: 194. Time: 13
Inserted 10000 records. n: 195. Time: 12
Inserted 10000 records. n: 196. Time: 13
Inserted 10000 records. n: 197. Time: 11
Inserted 10000 records. n: 198. Time: 11
Inserted 10000 records. n: 199. Time: 12
Inserted 10000 records. n: 200. Time: 11
Inserted 10000 records. n: 201. Time: 11
Inserted 10000 records. n: 202. Time: 11
Inserted 10000 records. n: 203. Time: 11
Inserted 10000 records. n: 204. Time: 11
Inserted 10000 records. n: 205. Time: 12
Inserted 10000 records. n: 206. Time: 11
Inserted 10000 records. n: 207. Time: 10
Inserted 10000 records. n: 208. Time: 10
Inserted 10000 records. n: 209. Time: 11
Inserted 10000 records. n: 210. Time: 10
Inserted 10000 records. n: 211. Time: 10
Inserted 10000 records. n: 212. Time: 11
Inserted 10000 records. n: 213. Time: 11
Inserted 10000 records. n: 214. Time: 10
Inserted 10000 records. n: 215. Time: 11
Inserted 10000 records. n: 216. Time: 11
Inserted 10000 records. n: 217. Time: 12
Inserted 10000 records. n: 218. Time: 12
Inserted 10000 records. n: 219. Time: 12
Inserted 10000 records. n: 220. Time: 11
Inserted 10000 records. n: 221. Time: 12
Inserted 10000 records. n: 222. Time: 10
Inserted 10000 records. n: 223. Time: 11
Inserted 10000 records. n: 224. Time: 11
Inserted 10000 records. n: 225. Time: 11
Inserted 10000 records. n: 226. Time: 11
Inserted 10000 records. n: 227. Time: 11
Inserted 10000 records. n: 228. Time: 11
Inserted 10000 records. n: 229. Time: 11
Inserted 10000 records. n: 230. Time: 11
Inserted 10000 records. n: 231. Time: 10
Inserted 10000 records. n: 232. Time: 11
Inserted 10000 records. n: 233. Time: 10
Inserted 10000 records. n: 234. Time: 11
Inserted 10000 records. n: 235. Time: 11
Inserted 10000 records. n: 236. Time: 11
Inserted 10000 records. n: 237. Time: 11
Inserted 10000 records. n: 238. Time: 10
Inserted 10000 records. n: 239. Time: 11
Inserted 10000 records. n: 240. Time: 11
Inserted 10000 records. n: 241. Time: 11
Inserted 10000 records. n: 242. Time: 11
Inserted 10000 records. n: 243. Time: 11
Inserted 10000 records. n: 244. Time: 11
Inserted 10000 records. n: 245. Time: 11
Inserted 10000 records. n: 246. Time: 12
Inserted 10000 records. n: 247. Time: 11
Inserted 10000 records. n: 248. Time: 12
Inserted 10000 records. n: 249. Time: 12
Inserted 10000 records. n: 250. Time: 11
Inserted 10000 records. n: 251. Time: 12
Inserted 10000 records. n: 252. Time: 12
Inserted 10000 records. n: 253. Time: 13
Inserted 10000 records. n: 254. Time: 10
Inserted 10000 records. n: 255. Time: 12
Inserted 10000 records. n: 256. Time: 12
Inserted 10000 records. n: 257. Time: 13
Inserted 10000 records. n: 258. Time: 11
Inserted 10000 records. n: 259. Time: 10
Inserted 10000 records. n: 260. Time: 12
Inserted 10000 records. n: 261. Time: 11
Inserted 10000 records. n: 262. Time: 11
Inserted 10000 records. n: 263. Time: 12
Inserted 10000 records. n: 264. Time: 10
Inserted 10000 records. n: 265. Time: 10
Inserted 10000 records. n: 266. Time: 11
Inserted 10000 records. n: 267. Time: 10
Inserted 10000 records. n: 268. Time: 11
Inserted 10000 records. n: 269. Time: 12
Inserted 10000 records. n: 270. Time: 12
Inserted 10000 records. n: 271. Time: 11
Inserted 10000 records. n: 272. Time: 11
Inserted 10000 records. n: 273. Time: 10
Inserted 10000 records. n: 274. Time: 12
Inserted 10000 records. n: 275. Time: 9
Inserted 10000 records. n: 276. Time: 12
Inserted 10000 records. n: 277. Time: 11
Inserted 10000 records. n: 278. Time: 11
Inserted 10000 records. n: 279. Time: 11
Inserted 10000 records. n: 280. Time: 10
Inserted 10000 records. n: 281. Time: 11
Inserted 10000 records. n: 282. Time: 11
Inserted 10000 records. n: 283. Time: 10
Inserted 10000 records. n: 284. Time: 10
Inserted 10000 records. n: 285. Time: 11
Inserted 10000 records. n: 286. Time: 10
Inserted 10000 records. n: 287. Time: 10
Inserted 10000 records. n: 288. Time: 11
Inserted 10000 records. n: 289. Time: 10
Inserted 10000 records. n: 290. Time: 9
Inserted 10000 records. n: 291. Time: 16
Inserted 10000 records. n: 292. Time: 10
Inserted 10000 records. n: 293. Time: 9
Inserted 10000 records. n: 294. Time: 11
Inserted 10000 records. n: 295. Time: 9
Inserted 10000 records. n: 296. Time: 9
Inserted 10000 records. n: 297. Time: 11
Inserted 10000 records. n: 298. Time: 39
Inserted 10000 records. n: 299. Time: 52
Inserted 10000 records. n: 300. Time: 54
Inserted 10000 records. n: 301. Time: 42
Inserted 10000 records. n: 302. Time: 42
Inserted 10000 records. n: 303. Time: 11
Inserted 10000 records. n: 304. Time: 9
Inserted 10000 records. n: 305. Time: 7
Inserted 10000 records. n: 306. Time: 8
Inserted 10000 records. n: 307. Time: 10
Inserted 10000 records. n: 308. Time: 10
Inserted 10000 records. n: 309. Time: 10
Inserted 10000 records. n: 310. Time: 10
Inserted 10000 records. n: 311. Time: 9
Inserted 10000 records. n: 312. Time: 11
Inserted 10000 records. n: 313. Time: 11
Inserted 10000 records. n: 314. Time: 13
Inserted 10000 records. n: 315. Time: 13
Inserted 10000 records. n: 316. Time: 12
Inserted 10000 records. n: 317. Time: 11
Inserted 10000 records. n: 318. Time: 10
Inserted 10000 records. n: 319. Time: 9
Inserted 10000 records. n: 320. Time: 12
Inserted 10000 records. n: 321. Time: 11
Inserted 10000 records. n: 322. Time: 12
Inserted 10000 records. n: 323. Time: 10
Inserted 10000 records. n: 324. Time: 11
Inserted 10000 records. n: 325. Time: 11
Inserted 10000 records. n: 326. Time: 13
Inserted 10000 records. n: 327. Time: 13
Inserted 10000 records. n: 328. Time: 12
Inserted 10000 records. n: 329. Time: 12
Inserted 10000 records. n: 330. Time: 10
Inserted 10000 records. n: 331. Time: 11
Inserted 10000 records. n: 332. Time: 10
Inserted 10000 records. n: 333. Time: 11
Inserted 10000 records. n: 334. Time: 11
Inserted 10000 records. n: 335. Time: 11
Inserted 10000 records. n: 336. Time: 10
Inserted 10000 records. n: 337. Time: 11
Inserted 10000 records. n: 338. Time: 12
Inserted 10000 records. n: 339. Time: 13
Inserted 10000 records. n: 340. Time: 14
Inserted 10000 records. n: 341. Time: 13
Inserted 10000 records. n: 342. Time: 10
Inserted 10000 records. n: 343. Time: 10
Inserted 10000 records. n: 344. Time: 9
Inserted 10000 records. n: 345. Time: 11
Inserted 10000 records. n: 346. Time: 12
Inserted 10000 records. n: 347. Time: 12
Inserted 10000 records. n: 348. Time: 10
Inserted 10000 records. n: 349. Time: 11
Inserted 10000 records. n: 350. Time: 13
Inserted 10000 records. n: 351. Time: 13
Inserted 10000 records. n: 352. Time: 11
Inserted 10000 records. n: 353. Time: 12
Inserted 10000 records. n: 354. Time: 10
Inserted 10000 records. n: 355. Time: 10
Inserted 10000 records. n: 356. Time: 12
Inserted 10000 records. n: 357. Time: 11
Inserted 10000 records. n: 358. Time: 11
Inserted 10000 records. n: 359. Time: 12
Inserted 10000 records. n: 360. Time: 11
Inserted 10000 records. n: 361. Time: 12
Inserted 10000 records. n: 362. Time: 13
Inserted 10000 records. n: 363. Time: 12
Inserted 10000 records. n: 364. Time: 11
Inserted 10000 records. n: 365. Time: 11
Inserted 10000 records. n: 366. Time: 11
Inserted 10000 records. n: 367. Time: 11
Inserted 10000 records. n: 368. Time: 12
Inserted 10000 records. n: 369. Time: 11
Inserted 10000 records. n: 370. Time: 14
Inserted 10000 records. n: 371. Time: 14
Inserted 10000 records. n: 372. Time: 15
Inserted 10000 records. n: 373. Time: 16
Inserted 10000 records. n: 374. Time: 18
Inserted 10000 records. n: 375. Time: 15
Inserted 10000 records. n: 376. Time: 11
Inserted 10000 records. n: 377. Time: 11
Inserted 10000 records. n: 378. Time: 11
Inserted 10000 records. n: 379. Time: 12
Inserted 10000 records. n: 380. Time: 11
Inserted 10000 records. n: 381. Time: 12
Inserted 10000 records. n: 382. Time: 12
Inserted 10000 records. n: 383. Time: 14
Inserted 10000 records. n: 384. Time: 12
Inserted 10000 records. n: 385. Time: 16
Inserted 10000 records. n: 386. Time: 14
Inserted 10000 records. n: 387. Time: 10
Inserted 10000 records. n: 388. Time: 12
Inserted 10000 records. n: 389. Time: 11
Inserted 10000 records. n: 390. Time: 11
Inserted 10000 records. n: 391. Time: 11
Inserted 10000 records. n: 392. Time: 12
Inserted 10000 records. n: 393. Time: 15
Inserted 10000 records. n: 394. Time: 16
Inserted 10000 records. n: 395. Time: 17
Inserted 10000 records. n: 396. Time: 13
Inserted 10000 records. n: 397. Time: 13
Inserted 10000 records. n: 398. Time: 17
Inserted 10000 records. n: 399. Time: 11
Inserted 10000 records. n: 400. Time: 16
Inserted 10000 records. n: 401. Time: 12
Inserted 10000 records. n: 402. Time: 13
Inserted 10000 records. n: 403. Time: 13
Inserted 10000 records. n: 404. Time: 13
Inserted 10000 records. n: 405. Time: 16
Inserted 10000 records. n: 406. Time: 10
Inserted 10000 records. n: 407. Time: 12
Inserted 10000 records. n: 408. Time: 14
Inserted 10000 records. n: 409. Time: 12
Inserted 10000 records. n: 410. Time: 18
Inserted 10000 records. n: 411. Time: 13
Inserted 10000 records. n: 412. Time: 20
Inserted 10000 records. n: 413. Time: 14
Inserted 10000 records. n: 414. Time: 15
Inserted 10000 records. n: 415. Time: 52
Inserted 10000 records. n: 416. Time: 22
Inserted 10000 records. n: 417. Time: 111
Inserted 10000 records. n: 418. Time: 81
Inserted 10000 records. n: 419. Time: 80
Inserted 10000 records. n: 420. Time: 78
Inserted 10000 records. n: 421. Time: 116
Inserted 10000 records. n: 422. Time: 123
Inserted 10000 records. n: 423. Time: 86
Inserted 10000 records. n: 424. Time: 24
Inserted 10000 records. n: 425. Time: 36
Inserted 10000 records. n: 426. Time: 21
Inserted 10000 records. n: 427. Time: 15
Inserted 10000 records. n: 428. Time: 17
Inserted 10000 records. n: 429. Time: 15
Inserted 10000 records. n: 430. Time: 15
Inserted 10000 records. n: 431. Time: 16
Inserted 10000 records. n: 432. Time: 15
Inserted 10000 records. n: 433. Time: 18
Inserted 10000 records. n: 434. Time: 20
Inserted 10000 records. n: 435. Time: 18
Inserted 10000 records. n: 436. Time: 20
Inserted 10000 records. n: 437. Time: 20
Inserted 10000 records. n: 438. Time: 22
Inserted 10000 records. n: 439. Time: 22
Inserted 10000 records. n: 440. Time: 26
Inserted 10000 records. n: 441. Time: 24
Inserted 10000 records. n: 442. Time: 69
Inserted 10000 records. n: 443. Time: 133
Inserted 10000 records. n: 444. Time: 17
Inserted 10000 records. n: 445. Time: 18
Inserted 10000 records. n: 446. Time: 20
Inserted 10000 records. n: 447. Time: 19
Inserted 10000 records. n: 448. Time: 19
Inserted 10000 records. n: 449. Time: 20
Inserted 10000 records. n: 450. Time: 18
Inserted 10000 records. n: 451. Time: 18
Inserted 10000 records. n: 452. Time: 19
Inserted 10000 records. n: 453. Time: 19
Inserted 10000 records. n: 454. Time: 19
Inserted 10000 records. n: 455. Time: 22
Inserted 10000 records. n: 456. Time: 33
Inserted 10000 records. n: 457. Time: 25
Inserted 10000 records. n: 458. Time: 23
Inserted 10000 records. n: 459. Time: 24
Inserted 10000 records. n: 460. Time: 134
Inserted 10000 records. n: 461. Time: 31
Inserted 10000 records. n: 462. Time: 21
Inserted 10000 records. n: 463. Time: 21
Inserted 10000 records. n: 464. Time: 21
Inserted 10000 records. n: 465. Time: 20
Inserted 10000 records. n: 466. Time: 25
Inserted 10000 records. n: 467. Time: 26
Inserted 10000 records. n: 468. Time: 24
Inserted 10000 records. n: 469. Time: 25
Inserted 10000 records. n: 470. Time: 31
Inserted 10000 records. n: 471. Time: 28
Inserted 10000 records. n: 472. Time: 31
Inserted 10000 records. n: 473. Time: 32
Inserted 10000 records. n: 474. Time: 33
Inserted 10000 records. n: 475. Time: 31
Inserted 10000 records. n: 476. Time: 40
Inserted 10000 records. n: 477. Time: 120
Inserted 10000 records. n: 478. Time: 23
Inserted 10000 records. n: 479. Time: 30
Inserted 10000 records. n: 480. Time: 33
Inserted 10000 records. n: 481. Time: 28
Inserted 10000 records. n: 482. Time: 26
Inserted 10000 records. n: 483. Time: 31
Inserted 10000 records. n: 484. Time: 22
Inserted 10000 records. n: 485. Time: 26
Inserted 10000 records. n: 486. Time: 25
Inserted 10000 records. n: 487. Time: 22
Inserted 10000 records. n: 488. Time: 38
Inserted 10000 records. n: 489. Time: 24
Inserted 10000 records. n: 490. Time: 27
Inserted 10000 records. n: 491. Time: 42
Inserted 10000 records. n: 492. Time: 39
Inserted 10000 records. n: 493. Time: 97
Inserted 10000 records. n: 494. Time: 59
Inserted 10000 records. n: 495. Time: 28
Inserted 10000 records. n: 496. Time: 24
Inserted 10000 records. n: 497. Time: 25
Inserted 10000 records. n: 498. Time: 24
Inserted 10000 records. n: 499. Time: 37
Inserted 10000 records. n: 500. Time: 24
Inserted 10000 records. n: 501. Time: 30
Inserted 10000 records. n: 502. Time: 31
Inserted 10000 records. n: 503. Time: 38
Inserted 10000 records. n: 504. Time: 65
Inserted 10000 records. n: 505. Time: 61
Inserted 10000 records. n: 506. Time: 47
Inserted 10000 records. n: 507. Time: 31
Inserted 10000 records. n: 508. Time: 38
Inserted 10000 records. n: 509. Time: 29
Inserted 10000 records. n: 510. Time: 31
Inserted 10000 records. n: 511. Time: 30
Inserted 10000 records. n: 512. Time: 33
Inserted 10000 records. n: 513. Time: 30
Inserted 10000 records. n: 514. Time: 35
Inserted 10000 records. n: 515. Time: 35
Inserted 10000 records. n: 516. Time: 33
Inserted 10000 records. n: 517. Time: 33
Inserted 10000 records. n: 518. Time: 30
Inserted 10000 records. n: 519. Time: 46
Inserted 10000 records. n: 520. Time: 75
Inserted 10000 records. n: 521. Time: 30
Inserted 10000 records. n: 522. Time: 36
Inserted 10000 records. n: 523. Time: 28
Inserted 10000 records. n: 524. Time: 29
Inserted 10000 records. n: 525. Time: 32
Inserted 10000 records. n: 526. Time: 34
Inserted 10000 records. n: 527. Time: 43
Inserted 10000 records. n: 528. Time: 38
Inserted 10000 records. n: 529. Time: 37
Inserted 10000 records. n: 530. Time: 29
Inserted 10000 records. n: 531. Time: 29
Inserted 10000 records. n: 532. Time: 30
Inserted 10000 records. n: 533. Time: 33
Inserted 10000 records. n: 534. Time: 39
Inserted 10000 records. n: 535. Time: 55
Inserted 10000 records. n: 536. Time: 73
Inserted 10000 records. n: 537. Time: 50
Inserted 10000 records. n: 538. Time: 54
Inserted 10000 records. n: 539. Time: 31
Inserted 10000 records. n: 540. Time: 38
Inserted 10000 records. n: 541. Time: 50
Inserted 10000 records. n: 542. Time: 40
Inserted 10000 records. n: 543. Time: 21
Inserted 10000 records. n: 544. Time: 38
Inserted 10000 records. n: 545. Time: 34
Inserted 10000 records. n: 546. Time: 41
Inserted 10000 records. n: 547. Time: 34
Inserted 10000 records. n: 548. Time: 31
Inserted 10000 records. n: 549. Time: 37
Inserted 10000 records. n: 550. Time: 58
Inserted 10000 records. n: 551. Time: 42
Inserted 10000 records. n: 552. Time: 36
Inserted 10000 records. n: 553. Time: 38
Inserted 10000 records. n: 554. Time: 55
Inserted 10000 records. n: 555. Time: 39
Inserted 10000 records. n: 556. Time: 45
Inserted 10000 records. n: 557. Time: 36
Inserted 10000 records. n: 558. Time: 48
Inserted 10000 records. n: 559. Time: 45
Inserted 10000 records. n: 560. Time: 36
Inserted 10000 records. n: 561. Time: 32
Inserted 10000 records. n: 562. Time: 36
Inserted 10000 records. n: 563. Time: 46
Inserted 10000 records. n: 564. Time: 51
Inserted 10000 records. n: 565. Time: 49
Inserted 10000 records. n: 566. Time: 47
Inserted 10000 records. n: 567. Time: 49
Inserted 10000 records. n: 568. Time: 45
Inserted 10000 records. n: 569. Time: 41
Inserted 10000 records. n: 570. Time: 39
Inserted 10000 records. n: 571. Time: 43
Inserted 10000 records. n: 572. Time: 68
Inserted 10000 records. n: 573. Time: 41
Inserted 10000 records. n: 574. Time: 33
Inserted 10000 records. n: 575. Time: 54
Inserted 10000 records. n: 576. Time: 39
Inserted 10000 records. n: 577. Time: 45
Inserted 10000 records. n: 578. Time: 50
Inserted 10000 records. n: 579. Time: 34
Inserted 10000 records. n: 580. Time: 36
Inserted 10000 records. n: 581. Time: 47
Inserted 10000 records. n: 582. Time: 59
Inserted 10000 records. n: 583. Time: 40
Inserted 10000 records. n: 584. Time: 37
Inserted 10000 records. n: 585. Time: 35
Inserted 10000 records. n: 586. Time: 59
Inserted 10000 records. n: 587. Time: 62
Inserted 10000 records. n: 588. Time: 26
Inserted 10000 records. n: 589. Time: 54
Inserted 10000 records. n: 590. Time: 54
Inserted 10000 records. n: 591. Time: 47
Inserted 10000 records. n: 592. Time: 53
Inserted 10000 records. n: 593. Time: 41
Inserted 10000 records. n: 594. Time: 45
Inserted 10000 records. n: 595. Time: 39
Inserted 10000 records. n: 596. Time: 45
Inserted 10000 records. n: 597. Time: 55
Inserted 10000 records. n: 598. Time: 50
Inserted 10000 records. n: 599. Time: 47
Inserted 10000 records. n: 600. Time: 39
Inserted 10000 records. n: 601. Time: 48
Inserted 10000 records. n: 602. Time: 43
Inserted 10000 records. n: 603. Time: 57
Inserted 10000 records. n: 604. Time: 38
Inserted 10000 records. n: 605. Time: 42
Inserted 10000 records. n: 606. Time: 44
Inserted 10000 records. n: 607. Time: 55
Inserted 10000 records. n: 608. Time: 46
Inserted 10000 records. n: 609. Time: 49
Inserted 10000 records. n: 610. Time: 44
Inserted 10000 records. n: 611. Time: 43
Inserted 10000 records. n: 612. Time: 43
Inserted 10000 records. n: 613. Time: 48
Inserted 10000 records. n: 614. Time: 36
Inserted 10000 records. n: 615. Time: 55
Inserted 10000 records. n: 616. Time: 58
Inserted 10000 records. n: 617. Time: 39
Is there any way to keep the performance linear at around 9-10 seconds per 10000 records?
Your calculations concentrate on the key side (Will the keys fit into RAM?). That is important, but after reaching some database size (which is dominated by the value side in your case), the values written to disk will become more relevant for speed. I fear that you can only optimize your disk (any details?) in such a situation.
Did you modify msiz too?
Did you try a lower bnum parameter (something like 1 or 2 Billions)?
What is your filesystem (ext4 performed far better than ext3 during my tests)?
Related
Loop though each observation in SAS
Let say I have a table of 10000 observations: Obs X Y Z 1 2 3 ... 10000 For each observation, I create a macro: mymacro(X, Y, Z) where I use X, Y, Z like inputs. My macro create a table with 1 observation, 4 new variables var1, var2, var3, var4. I would like to know how to loop through 10000 observations in my initial set, and the result would be like: Obs X Y Z Var1 Var2 Var3 Var4 1 2 3 ... 10000 Update: The calculation of Var1, Var2, Var3, Var4: I have a reference table: Z 25 26 27 28 29 30 0 10 000 10 000 10 000 10 000 10 000 10 000 1 10 000 10 000 10 000 10 000 10 000 10 000 2 10 000 10 000 10 000 10 000 10 000 10 000 3 10 000 10 000 10 000 10 000 10 000 10 000 4 9 269 9 322 9 322 9 381 9 381 9 436 5 8 508 8 619 8 619 8 743 8 743 8 850 6 7 731 7 914 7 914 8 102 8 102 8 258 7 6 805 7 040 7 040 7 280 7 280 7 484 8 5 864 6 137 6 137 6 421 6 421 6 655 9 5 025 5 328 5 328 5 629 5 629 5 929 10 4 359 4 648 4 648 4 934 4 934 5 320 And my have set is like: Obs X Y Z 1 27 4 9 2 3 10000 So for the first observation (27, 4, 9): Var1 = (8 619+ 7 914+ 7 040 + 6 137 + 5 328)/ 9 322 Var2 = (8 743+ 8 102+ 7 280+ 6 421 + 5 629 )/ 9 381 So that: Var1 = Sum of all number in column 27 (X), from the observation 5 (Z+1) to the observation 9 (Z), and divided by the value in the (column 27 (X) - observation 4 (Z)) Var2 = Sum of all number in column 28 (X+1), from the observation 5 (Z+1) to the observation 9 (Z), and divided by the value in the (column 28 (X+1) - observation 4 (Z))
I would convert the reference table to a form that lets you do the calculations for all observations at once. So make your reference table into a tall structure, either by transposing the existing table or just reading it that way to start with: data ref_tall; input z #; do col=25 to 30 ; input value :comma9. #; output; end; datalines; 0 10,000 10,000 10,000 10,000 10,000 10,000 1 10,000 10,000 10,000 10,000 10,000 10,000 2 10,000 10,000 10,000 10,000 10,000 10,000 3 10,000 10,000 10,000 10,000 10,000 10,000 4 9,269 9,322 9,322 9,381 9,381 9,436 5 8,508 8,619 8,619 8,743 8,743 8,850 6 7,731 7,914 7,914 8,102 8,102 8,258 7 6,805 7,040 7,040 7,280 7,280 7,484 8 5,864 6,137 6,137 6,421 6,421 6,655 9 5,025 5,328 5,328 5,629 5,629 5,929 10 4,359 4,648 4,648 4,934 4,934 5,320 ; Now take your list table HAVE: data have; input id x y z; datalines; 1 27 4 9 2 25 2 4 ; And combine it with the reference table and make your calculations: proc sql ; create table want1 as select a.id , sum(b.value)/min(c.value) as var1 from have a left join ref_tall b on a.x=b.col and b.z between a.y+1 and a.z left join ref_tall c on a.x=c.col and c.z = a.y group by a.id ; create table want2 as select a.id , sum(d.value)/min(e.value) as var2 from have a left join ref_tall d on a.x+1=d.col and d.z between a.y+1 and a.z left join ref_tall e on a.x+1=e.col and e.z = a.y group by a.id ; create table want as select * from want1 natural join want2 natural join have ; quit; Results: Obs id x y z var1 var2 1 1 27 4 9 3.75864 3.85620 2 2 25 2 4 1.92690 1.93220
The reference table can be established in an array that makes performing the specified computations easy. The reference values can than be accessed using a direct address reference. Example The reference table data was moved into a data set so the values can be changed over time or reloaded from some source such as Excel. The reference values can be loaded into an array for use during a DATA step. * reference information in data set, x property column names are _<num>; data ref; input z (_25-_30) (comma9. &); datalines; 0 10,000 10,000 10,000 10,000 10,000 10,000 1 10,000 10,000 10,000 10,000 10,000 10,000 2 10,000 10,000 10,000 10,000 10,000 10,000 3 10,000 10,000 10,000 10,000 10,000 10,000 4 9,269 9,322 9,322 9,381 9,381 9,436 5 8,508 8,619 8,619 8,743 8,743 8,850 6 7,731 7,914 7,914 8,102 8,102 8,258 7 6,805 7,040 7,040 7,280 7,280 7,484 8 5,864 6,137 6,137 6,421 6,421 6,655 9 5,025 5,328 5,328 5,629 5,629 5,929 10 4,359 4,648 4,648 4,934 4,934 5,320 ; * computation parameters, might be a thousand of them specified; data have; input id x y z; datalines; 1 27 4 9 ; * perform computation for each parameters specified; data want; set have; array ref[0:10,1:30] _temporary_; if _n_ = 1 then do ref_row = 0 by 1 until (last_ref); * load reference data into an array for direct addressing during computation; set ref end=last_ref; array ref_cols _25-_30; do index = 1 to dim(ref_cols); colname = vname(ref_cols[index]); colnum = input(substr(colname,2),8.); ref[ref_row,colnum] = ref_cols[index]; end; end; * perform computation for parameters specified; array vars var1-var4; do index = 1 to dim(vars); ref_column = x + index - 1 ; * column x, then x+1, then x+2, then x+3; numerator = 0; * algorithm against reference data; do ref_row = y+1 to z; numerator + ref[ref_row,ref_column]; end; denominator = ref[y,ref_column]; vars[index] = numerator / denominator; * result; end; keep id x y z numerator denominator var1-var4; run;
Calculating a column in SQL using the column's own output as input
I have problem that I find very hard to solve: I need to calculate a column R_t in SQL where for each row, the sum of the "previous" calculated values SUM(R_t-1) is required as input. The calculation is done grouped over a ProjectID column. I have no clue how to proceed. The formula for the calculation I am trying to achieve is R_t = ([Contract value]t - SUM(R{t-1})) / [Remaining Hours]_t * [HoursRegistered]t where "t" denotes time and SUM(R{t-1}) is the sum of R_t from t = 0 to t-1. Time is always consecutive and always begin in t = 0. But number of time periods may differ across [ProjectID], i.e. one project having t = {0,1,2} and another t = {0,1,2,3,4,5}. The time period will never "jump" from 5 to 7 The expected output (using the data from below is) for ProjectID 101 is R_0 = (500,000 - 0) / 500 * 65 = 65,000 R_1 = (500,000 - (65,000)) / 435 * 100 = 100,000 R_2 = (500,000 - (65,000 + 100,000)) / 335 * 85 = 85,000 R_3 = (500,000 - (65,000 + 100,000 + 85,000)) / 250 * 69 = 69,000 etc... This calculation is done for each ProjectID. My question is how to formulate this in a SQL query? My first thought was to create a recursive CTE, but I am actually not sure it is the right way proceed. Recursive CTE is (from my understanding) made for handling more of hierarchical like structure, which this isn't really. My other thought was to calculate the SUM(R_t-1) using windowed functions, ie SUM OVER (PARITION BY ORDER BY) with a LAG, but the recursiveness really gives me trouble and I run my head against the wall when I am trying. Below a query for creating the input data CREATE TABLE [dbo].[InputForRecursiveCalculation] ( [Time] int NULL, ProjectID [int], ContractValue float, ContractHours float, HoursRegistered float, RemainingHours float ) GO INSERT INTO [dbo].[InputForRecursiveCalculation] ( [Time] ,[ProjectID] ,[ContractValue] ,[ContractHours] ,[HoursRegistered] ,[RemainingHours] ) VALUES (0,101,500000,500,65,500), (1,101,500000,500,100,435), (2,101,500000,500,85,335), (3,101,500000,500,69,250), (4,101,450000,650,100,331), (5,101,450000,650,80,231), (6,101,450000,650,90,151), (7,101,450000,650,45,61), (8,101,450000,650,16,16), (0,110,120000,90,10,90), (1,110,120000,90,10,80), (2,110,130000,90,10,70), (3,110,130000,90,10,60), (4,110,130000,90,10,50), (5,110,130000,90,10,40), (6,110,130000,90,10,30), (7,110,130000,90,10,20), (8,110,130000,90,10,10) GO For those of you who dare downloading something from a complete stranger, I have created an Excel file demonstrating the calculation (please download the file as you will not be to see the actual formula in the HTML representation shown when first clicking the link): https://www.dropbox.com/s/3rxz72lbvooyc4y/Calculation%20example.xlsx?dl=0 Best regards, Victor
I think it will be usefull for you. There is additional column SumR that stands for sumarry of previest rows (for ProjectID) ;with recu as ( select Time, ProjectId, ContractValue, ContractHours, HoursRegistered, RemainingHours, cast((ContractValue - 0)*HoursRegistered/RemainingHours as numeric(15,0)) as R, cast((ContractValue - 0)*HoursRegistered/RemainingHours as numeric(15,0)) as SumR from InputForRecursiveCalculation where Time=0 union all select input.Time, input.ProjectId, input.ContractValue, input.ContractHours, input.HoursRegistered, input.RemainingHours, cast((input.ContractValue - prev.SumR)*input.HoursRegistered/input.RemainingHours as numeric(15,0)), cast((input.ContractValue - prev.SumR)*input.HoursRegistered/input.RemainingHours + prev.SumR as numeric(15,0)) from recu prev inner join InputForRecursiveCalculation input on input.ProjectId = prev.ProjectId and input.Time = prev.Time + 1 ) select * from recu order by ProjectID, Time RESULTS: Time ProjectId ContractValue ContractHours HoursRegistered RemainingHours R SumR ----------- ----------- ---------------------- ---------------------- ---------------------- ---------------------- --------------------------------------- --------------------------------------- 0 101 500000 500 65 500 65000 65000 1 101 500000 500 100 435 100000 165000 2 101 500000 500 85 335 85000 250000 3 101 500000 500 69 250 69000 319000 4 101 450000 650 100 331 39577 358577 5 101 450000 650 80 231 31662 390239 6 101 450000 650 90 151 35619 425858 7 101 450000 650 45 61 17810 443668 8 101 450000 650 16 16 6332 450000 0 110 120000 90 10 90 13333 13333 1 110 120000 90 10 80 13333 26666 2 110 130000 90 10 70 14762 41428 3 110 130000 90 10 60 14762 56190 4 110 130000 90 10 50 14762 70952 5 110 130000 90 10 40 14762 85714 6 110 130000 90 10 30 14762 100476 7 110 130000 90 10 20 14762 115238 8 110 130000 90 10 10 14762 130000
LARGE & SMALL from ARRAY by MATCH
Straight to the question. I want to filter K values by X matching Z. From filtered values of K find LARGE & SMALL of coresponding Y. So if X=10 than find large and small of value Y=1600 from list of values K with index Z=10. LARGE=1500 & SMALL=2000 X Y 5 3000 5 3200 5 2800 10 1600 10 2900 10 3500 15 1800 15 4500 Z K 5 1000 5 2000 5 3000 5 4000 10 1000 10 1500 10 2000 10 3000 10 3500 10 5000 15 1000 15 1500 15 2500 15 3000 15 4000 15 5500 TABLE IMAGE I cannot figure out how to filter array of K with X=Z.
For Large: =AGGREGATE(14,6,$G$3:$G$29/(($F$3:$F$29=A3)*($G$3:$G$29<=B3)),1) For Small =AGGREGATE(15,6,$G$3:$G$29/(($F$3:$F$29=A3)*($G$3:$G$29>=B3)),1)
Index on NUMBER column on Oracle 10g
currently we have a column with only integer values declared as NUMBER. At the same time it is our (only) index. I wonder if it would make a difference in performance if you declare the index as INTEGER? Or is Oracle smart enough to see that it is an integer? Thank you very much.
No, it won't. Taking Florin's test tables, you can set up a small test harness that runs each query hundreds of times and averages the elapsed time. In my case, I ran both queries 500 times each. Sometimes, the NUMBER version will run slightly faster (1.232 hundredths of a second vs 1.284 hundredths of a second). SQL> ed Wrote file afiedt.buf 1 declare 2 l_start_time number; 3 l_end_time number; 4 l_cnt number; 5 l_iterations number := 500; 6 begin 7 l_start_time := dbms_utility.get_time(); 8 for i in 1 .. l_iterations 9 loop 10 select count(*) 11 into l_cnt 12 from fg_test; 13 end loop; 14 l_end_time := dbms_utility.get_time(); 15 dbms_output.put_line( 'Average elapsed (number) = ' || 16 (l_end_time - l_start_time)/l_iterations || 17 ' hundredths of a second.' ); 18 l_start_time := dbms_utility.get_time(); 19 for i in 1 .. l_iterations 20 loop 21 select count(*) 22 into l_cnt 23 from fg_test1; 24 end loop; 25 l_end_time := dbms_utility.get_time(); 26 dbms_output.put_line( 'Average elapsed (integer) = ' || 27 (l_end_time - l_start_time)/l_iterations || 28 ' hundredths of a second.' ); 29* end; 30 / Average elapsed (number) = 1.232 hundredths of a second. Average elapsed (integer) = 1.284 hundredths of a second. PL/SQL procedure successfully completed. Elapsed: 00:00:12.60 If you immediately run the same code block again, however, you're just as likely to see the reverse where the integer version runs slightly faster. SQL> / Average elapsed (number) = 1.256 hundredths of a second. Average elapsed (integer) = 1.22 hundredths of a second. PL/SQL procedure successfully completed. Elapsed: 00:00:12.38 Realistically, where you're trying to measure differences in milliseconds or fractions of milliseconds, you're well into the realm where system noise is going to come into play. Even though my machine is "idle" other than the test I'm running, there are thousands of reasons why the system might add an extra millisecond or two to an elapsed time to deal with some interrupt or to run some background thread that does something for the operating system. This result makes sense when you consider that INTEGER is just a synonym for NUMBER(38) SQL> desc fg_test1; Name Null? Type ----------------------------------------- -------- ---------------------------- A NUMBER(38) SQL> desc fg_test; Name Null? Type ----------------------------------------- -------- ---------------------------- A NUMBER Update: Even using a NUMBER(6) (note that the INSERT has to be changed to load only 999,999 rows rather than 1 million), there is no change Create the table SQL> create table fg_test2(a number(6)); Table created. Elapsed: 00:00:00.01 SQL> ed Wrote file afiedt.buf 1 insert into fg_test2 2* select level from dual connect by level <= 1000000-1 SQL> / 999999 rows created. Elapsed: 00:00:07.61 SQL> create index fg_ix2 on fg_test2(a); Index created. Elapsed: 00:00:00.01 Run the script. Note that there are no significant differences across any of the four runs and (by chance) in none of the four cases is the NUMBER(6) table the most efficient. SQL> ed Wrote file afiedt.buf 1 declare 2 l_start_time number; 3 l_end_time number; 4 l_cnt number; 5 l_iterations number := 500; 6 begin 7 l_start_time := dbms_utility.get_time(); 8 for i in 1 .. l_iterations 9 loop 10 select count(*) 11 into l_cnt 12 from fg_test; 13 end loop; 14 l_end_time := dbms_utility.get_time(); 15 dbms_output.put_line( 'Average elapsed (number) = ' || 16 (l_end_time - l_start_time)/l_iterations || 17 ' hundredths of a second.' ); 18 l_start_time := dbms_utility.get_time(); 19 for i in 1 .. l_iterations 20 loop 21 select count(*) 22 into l_cnt 23 from fg_test1; 24 end loop; 25 l_end_time := dbms_utility.get_time(); 26 dbms_output.put_line( 'Average elapsed (integer) = ' || 27 (l_end_time - l_start_time)/l_iterations || 28 ' hundredths of a second.' ); 29 l_start_time := dbms_utility.get_time(); 30 for i in 1 .. l_iterations 31 loop 32 select count(*) 33 into l_cnt 34 from fg_test2; 35 end loop; 36 l_end_time := dbms_utility.get_time(); 37 dbms_output.put_line( 'Average elapsed (number(6)) = ' || 38 (l_end_time - l_start_time)/l_iterations || 39 ' hundredths of a second.' ); 40* end; SQL> / Average elapsed (number) = 1.236 hundredths of a second. Average elapsed (integer) = 1.234 hundredths of a second. Average elapsed (number(6)) = 1.306 hundredths of a second. PL/SQL procedure successfully completed. Elapsed: 00:00:18.89 SQL> / Average elapsed (number) = 1.208 hundredths of a second. Average elapsed (integer) = 1.228 hundredths of a second. Average elapsed (number(6)) = 1.312 hundredths of a second. PL/SQL procedure successfully completed. Elapsed: 00:00:18.74 SQL> / Average elapsed (number) = 1.208 hundredths of a second. Average elapsed (integer) = 1.232 hundredths of a second. Average elapsed (number(6)) = 1.288 hundredths of a second. PL/SQL procedure successfully completed. Elapsed: 00:00:18.66 SQL> / Average elapsed (number) = 1.21 hundredths of a second. Average elapsed (integer) = 1.22 hundredths of a second. Average elapsed (number(6)) = 1.292 hundredths of a second. PL/SQL procedure successfully completed. Elapsed: 00:00:18.62
UPDATE: My test had a small problem. ( I tried for first table to insert 10M rows, but connect by has raised an insufficient memory exception. However probly was inserted 2-3M rows and then rolbaked. So, my first table had same number of rows, but more blocks.) So, the assertions below are not verified. The answer is yes. (but how much you obtain from this, you should test with you critical operations.) INTEGER is a subtype of NUMBER. But, surprisingly, subtipes of NUMBER are allways faster(need link here). My test case: create table fg_test(a number); insert into fg_test select level from dual connect by level <= 1000000; --1000000 rows inserted create index fg_ix on fg_test(a); select count(*) from fg_test; -- >141 msecs create table fg_test1(a INTEGER); insert into fg_test1 select level from dual connect by level <= 1000000; --1000000 rows inserted create index fg_ix1 on fg_test1(a); select count(*) from fg_test1; -- > 116 msecs Explanation: select count(*) will do a fast full scan on the index. I ran the select count(*) muuuultiple times to see how what is best speed. In general, with INTEGER is faster. Best speed of INTEGER is better than best speed of NUMBER;
transact SQL, sum each row and insert into another table
for a table on ms-sql2000 containing the following columns and numbers: S_idJ_id Se_id B_id Status Count multiply 63 1000 16 12 1 10 2 64 1001 12 16 1 9 3 65 1002 17 12 1 10 2 66 1003 16 12 1 6 3 67 1004 12 16 1 10 2 I want to generate an classic asp script which will do the following for each row where status=1 : -multiply -> answer= multiply column 'count' with column 'multiply' Then: count the total answer and sum for each se_id like : se_id total 12 47 16 38 17 20 and display on screen like Rank se_id total 1 12 47 2 16 38 3 17 20 Condition: if there are multiple equal total values then give the lower numbered se_id a priority for getting a ranking and give the next higher numbered se_id the next number in rank Any sample code in classic asp or advice is welcome on how to get this accomplished
'score' = source table. if (EXISTS (select * from INFORMATION_SCHEMA.TABLES where TABLE_NAME = 'result_table')) begin drop table result_table; end select rank = IDENTITY(INT,1,1), se_id, sum(multiply * count) as total into result_table from score where status = 1 group by se_id order by total desc, se_id; [Edit] Change query as answer on first comment