+1
Hi,
I’d be interested in getting feedback from people that are already using HPE Apollo’s or other high density servers with local storage as repository server. It seems that a lot of people are currently planning to use them.
+1
Did you try a single-job (--numjobs=1) test? For read, I think, this is also interesting.
Here are numbers for write and read with one task.
# fio --rw=read --name=test --size=100G --direct=1 --bs=512k --numjobs=1
test: (g=0): rw=read, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=psync, iodepth=1
fio-3.19
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=2052MiB/s][r=4104 IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=110268: Tue May 11 14:34:11 2021
read: IOPS=4493, BW=2247MiB/s (2356MB/s)(100GiB/45572msec)
clat (usec): min=126, max=87993, avg=221.41, stdev=921.48
lat (usec): min=126, max=87994, avg=221.57, stdev=921.49
clat percentiles (usec):
| 1.00th=[ 128], 5.00th=[ 129], 10.00th=[ 130], 20.00th=[ 133],
| 30.00th=[ 143], 40.00th=[ 163], 50.00th=[ 167], 60.00th=[ 169],
| 70.00th=[ 169], 80.00th=[ 172], 90.00th=[ 172], 95.00th=[ 176],
| 99.00th=[ 502], 99.50th=[ 4752], 99.90th=[15008], 99.95th=[18744],
| 99.99th=[26346]
bw ( MiB/s): min= 906, max= 2671, per=100.00%, avg=2256.48, stdev=334.99, samples=90
iops : min= 1812, max= 5342, avg=4512.97, stdev=669.99, samples=90
lat (usec) : 250=98.46%, 500=0.54%, 750=0.03%, 1000=0.03%
lat (msec) : 2=0.10%, 4=0.23%, 10=0.42%, 20=0.14%, 50=0.04%
lat (msec) : 100=0.01%
cpu : usr=0.82%, sys=11.81%, ctx=204802, majf=0, minf=141
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=204800,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=2247MiB/s (2356MB/s), 2247MiB/s-2247MiB/s (2356MB/s-2356MB/s), io=100GiB (107GB), run=45572-45572msec
Disk stats (read/write):
sdb: ios=204355/0, merge=12/0, ticks=40212/0, in_queue=12267, util=99.95%
# fio --rw=write --name=test --size=100G --direct=1 --bs=512k --numjobs=1
test: (g=0): rw=write, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=psync, iodepth=1
fio-3.19
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][w=2330MiB/s][w=4659 IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=110600: Tue May 11 14:36:35 2021
write: IOPS=4684, BW=2342MiB/s (2456MB/s)(100GiB/43718msec); 0 zone resets
clat (usec): min=131, max=3000, avg=200.27, stdev=49.15
lat (usec): min=136, max=3016, avg=212.75, stdev=50.33
clat percentiles (usec):
| 1.00th=[ 137], 5.00th=[ 139], 10.00th=[ 153], 20.00th=[ 169],
| 30.00th=[ 174], 40.00th=[ 180], 50.00th=[ 192], 60.00th=[ 202],
| 70.00th=[ 212], 80.00th=[ 227], 90.00th=[ 253], 95.00th=[ 289],
| 99.00th=[ 375], 99.50th=[ 416], 99.90th=[ 529], 99.95th=[ 570],
| 99.99th=[ 816]
bw ( MiB/s): min= 2268, max= 2598, per=100.00%, avg=2346.06, stdev=80.44, samples=87
iops : min= 4536, max= 5196, avg=4692.13, stdev=160.85, samples=87
lat (usec) : 250=89.58%, 500=10.27%, 750=0.14%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=6.12%, sys=9.37%, ctx=204800, majf=0, minf=12
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,204800,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=2342MiB/s (2456MB/s), 2342MiB/s-2342MiB/s (2456MB/s-2456MB/s), io=100GiB (107GB), run=43718-43718msec
Disk stats (read/write):
sdb: ios=0/204799, merge=0/12, ticks=0/37238, in_queue=8, util=99.94%
+13
Thanks!
Very interesting! I got <300MB/sec with Windows and IOmeter with one worker reading.
+13
I would say, quite good performance!
Is this what you expected?
+1
I looked into io scheduler and wanted to switch to noop, but it’s not available in RHEL 8. I did not touch io-queue depth yet.
# cat /sys/block/sdf/queue/nr_requests
1013
+13
Well, depending on the test parameter I can reach 2,5GB/s. I mean… we probably will not be able to saturate this anyway in the near future. I just want to be sure that my RAID and fs setup correct as this is hard to modify once data is on the disk.
# fio --rw=write --name=test --size=50G --direct=1 --bs=512k --numjobs=20 --ioengine=libaio --iodepth=20 --refill_buffers --group_reporting
Run status group 0 (all jobs):
WRITE: bw=2463MiB/s (2582MB/s), 2463MiB/s-2463MiB/s (2582MB/s-2582MB/s), io=1000GiB (1074GB), run=415836-415836msec
Disk stats (read/write):
sdb: ios=0/2047239, merge=0/119, ticks=0/165897764, in_queue=164873511, util=100.00%
I think, you can sleep without worries 
Do you see average latency during tests?
+1
Well, depending on the test parameter I can reach 2,5GB/s. I mean… we probably will not be able to saturate this anyway in the near future. I just want to be sure that my RAID and fs setup correct as this is hard to modify once data is on the disk.
# fio --rw=write --name=test --size=50G --direct=1 --bs=512k --numjobs=20 --ioengine=libaio --iodepth=20 --refill_buffers --group_reporting
Run status group 0 (all jobs):
WRITE: bw=2463MiB/s (2582MB/s), 2463MiB/s-2463MiB/s (2582MB/s-2582MB/s), io=1000GiB (1074GB), run=415836-415836msec
Disk stats (read/write):
sdb: ios=0/2047239, merge=0/119, ticks=0/165897764, in_queue=164873511, util=100.00%
I think, you can sleep without worries
Do you see average latency during tests?
For the last test: lat (msec): min=4, max=434, avg=79.64, stdev=37.26
+13
Well, depending on the test parameter I can reach 2,5GB/s. I mean… we probably will not be able to saturate this anyway in the near future. I just want to be sure that my RAID and fs setup correct as this is hard to modify once data is on the disk.
# fio --rw=write --name=test --size=50G --direct=1 --bs=512k --numjobs=20 --ioengine=libaio --iodepth=20 --refill_buffers --group_reporting
Run status group 0 (all jobs):
WRITE: bw=2463MiB/s (2582MB/s), 2463MiB/s-2463MiB/s (2582MB/s-2582MB/s), io=1000GiB (1074GB), run=415836-415836msec
Disk stats (read/write):
sdb: ios=0/2047239, merge=0/119, ticks=0/165897764, in_queue=164873511, util=100.00%
I think, you can sleep without worries
Do you see average latency during tests?
For the last test: lat (msec): min=4, max=434, avg=79.64, stdev=37.26
Oh, average is quite high.
You may can reduce it by slightly decreasing queue depth. Probably performance will not change that much if reduction is small. I would try --iodepth=16
+1
Well, depending on the test parameter I can reach 2,5GB/s. I mean… we probably will not be able to saturate this anyway in the near future. I just want to be sure that my RAID and fs setup correct as this is hard to modify once data is on the disk.
# fio --rw=write --name=test --size=50G --direct=1 --bs=512k --numjobs=20 --ioengine=libaio --iodepth=20 --refill_buffers --group_reporting
Run status group 0 (all jobs):
WRITE: bw=2463MiB/s (2582MB/s), 2463MiB/s-2463MiB/s (2582MB/s-2582MB/s), io=1000GiB (1074GB), run=415836-415836msec
Disk stats (read/write):
sdb: ios=0/2047239, merge=0/119, ticks=0/165897764, in_queue=164873511, util=100.00%
I think, you can sleep without worries
Do you see average latency during tests?
For the last test: lat (msec): min=4, max=434, avg=79.64, stdev=37.26
Oh, average is quite high.
You may can reduce it by slightly decreasing queue depth. Probably performance will not change that much if reduction is small. I would try --iodepth=16
Similar throughput but avg latency below 50ms then.
+13
Sounds good! 50ms on SATA disks I would call “acceptable”. And this is near saturation; as you already said, you will probably will not see such counters in real life.
+13
Thanks are due to
+13
I’ve been watching this silently for a while but just want to say a huge thank you to both
It’s rare that we as a collective can all see real world numbers outside of specific lab simulations and free from marketing so this has been truly awesome to see. I’m still looking at potentially a Dell PowerEdge XE7100 based system so if I can organise a test I’d love to re-run these benchmarks and get a true Dell vs HPE battle going here!
+1
I wanted to post some Veeam benchmarks too, but this doesn’t make much sense as I can see that jobs are limited by network. We have a 40/50 GbE 547FLR network adapter with QSFP → SFP adapter, so this is currently limited by 2x10 GbE bonded linux interfaces. I see ~1,3GB/s and Veeam is showing network or source as bottleneck. Our network equipment will be replaced in the next months, then I will switch to 40GbE. As we are testing with v10, I also can not use the Apollos as proxy to backup directly from storage snapshot. Our existing proxies are sending the data to the Apollo via network.
One advice: make a large /tmp fs is you use a dedicated partition or lvm volume. Veeam is writing a lot to /tmp and my first jobs failed because it was only 4GB.
The Apollos are not deployed, we are currently installing them with RHEL 8.2. It will take a couple of days until I’ll be able to test.
just curious, why in your case did you choose RHEL 8.2 , instead of (e.g.) Ubuntu?
+1
Just noticed that my last post was in the wrong thread. Here are the two options I’m thinking about. We’ll probably try out a backup and copy extent on each server.
+1
No more tests 
already migrated most of the jobs. I had some problems with sudo/ssh, some jobs failed a lot of data was left in /opt/veeam (configured in registry) and in the Veeam user home, /var is growing rapidly. At one point Veeam could not perform any task on one server as the Veeam user was not able to connect to his nfs home anymore (>100 defunct processes). After disabling sudo and using the root user this did not happen again, but it’s not so nice needing root for this to work properly.
As we are still on v10 this might be much better in v11 with persistent data mover. All of those are no Apollo issues, it’s Veeam + Linux. Performance in real world is still pretty good, I’m just struggling with the available repository task slots as 52 cores per server are be a bit too little with backup + copy + offload tasks. Those 2 Apollos were initially bought only as copy target, but - as usual - mission has changed and they are now also used for backup. We’ll have 2-4 additional Apollos shortly, then the task slot issue should also be solved.
+8
after all i’m now member of apollo gang :). Don’t forget like me to configure cache of the controller, by default it was 100% to read. The difference was HUGE. Interesting fact my processing rate quadruplet compared to my old dedup appliance on same backup scenario. I will continue my test to a specific restoration scenario to compare read performances on backup repo.
fio --rw=write --name=test --size=50G --direct=1 --bs=512k --numjobs=20 --ioengine=libaio --iodepth=16 --refill_buffers --group_reporting
test: (g=0): rw=write, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=libaio, iodepth=16
...
fio-3.19
Starting 20 processes
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
test: Laying out IO file (1 file / 51200MiB)
Jobs: 11 (f=10): [f(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(4),W(1),_(2),W(4)][99.5%][w=2398MiB/s][w=4795 IOPS][eta 00m:02s]
test: (groupid=0, jobs=20): err= 0: pid=276626: Fri Feb 18 12:28:06 2022
write: IOPS=5030, BW=2515MiB/s (2637MB/s)(1000GiB/407115msec); 0 zone resets
slat (usec): min=5, max=204138, avg=2829.68, stdev=8357.51
clat (usec): min=1612, max=905853, avg=60461.38, stdev=32858.38
lat (usec): min=1621, max=914383, avg=63291.66, stdev=33526.12
clat percentiles (msec):
| 1.00th=[ 35], 5.00th=[ 39], 10.00th=[ 41], 20.00th=[ 43],
| 30.00th=[ 45], 40.00th=[ 47], 50.00th=[ 50], 60.00th=[ 53],
| 70.00th=[ 57], 80.00th=[ 65], 90.00th=[ 90], 95.00th=[ 155],
| 99.00th=[ 186], 99.50th=[ 197], 99.90th=[ 215], 99.95th=[ 224],
| 99.99th=[ 241]
bw ( MiB/s): min= 1528, max= 4374, per=100.00%, avg=2521.54, stdev=17.08, samples=16126
iops : min= 3055, max= 8738, avg=5032.83, stdev=34.26, samples=16126
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.09%, 50=51.84%
lat (msec) : 100=39.01%, 250=9.04%, 500=0.01%, 750=0.01%, 1000=0.01%
cpu : usr=5.01%, sys=1.05%, ctx=2038785, majf=0, minf=77794
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,2048000,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
WRITE: bw=2515MiB/s (2637MB/s), 2515MiB/s-2515MiB/s (2637MB/s-2637MB/s), io=1000GiB (1074GB), run=407115-407115msec
Disk stats (read/write):
sda: ios=0/2046821, merge=0/125, ticks=0/103413090, in_queue=103413090, util=100.00%
+3
i use Dell PowerEdge R740, R730. The performance is Good.
For how many VM’ s and backup data?
20 VMs
+1
I’ve also 2 RAID 60 with 28 x 16TB. In the reference Apollo it’s a single SOBR with 2 extents (2 RAID volumes). I’m testing one of them. In the screenshot in the link you see that a single volume is performing at max of 3GB/s.
+13
I’ve also 2 RAID 60 with 28 x 16TB. In the reference Apollo it’s a single SOBR with 2 extents (2 RAID volumes). I’m testing one of them. In the screenshot in the link you see that a single volume is performing at max of 3GB/s.
Which screenshot do you mean?
+1
I’ve also 2 RAID 60 with 28 x 16TB. In the reference Apollo it’s a single SOBR with 2 extents (2 RAID volumes). I’m testing one of them. In the screenshot in the link you see that a single volume is performing at max of 3GB/s.
Which screenshot do you mean?
The link that was in my earlier post https://community.hpe.com/t5/Around-the-Storage-Block/The-fastest-ever-all-in-one-backup-solution-from-HPE-Storage-and/ba-p/7126052#.YJqQarUzZWJ → screenshot 4 → https://community.hpe.com/t5/image/serverpage/image-id/122029i134C52F696347813/image-size/large?v=v2&px=2000
+13
I’ve also 2 RAID 60 with 28 x 16TB. In the reference Apollo it’s a single SOBR with 2 extents (2 RAID volumes). I’m testing one of them. In the screenshot in the link you see that a single volume is performing at max of 3GB/s.
Which screenshot do you mean?
The link that was in my earlier post https://community.hpe.com/t5/Around-the-Storage-Block/The-fastest-ever-all-in-one-backup-solution-from-HPE-Storage-and/ba-p/7126052#.YJqQarUzZWJ → screenshot 4 → https://community.hpe.com/t5/image/serverpage/image-id/122029i134C52F696347813/image-size/large?v=v2&px=2000
Thanks, now I see your point: Purple and azure lines.
Did you try to rise io-queue depth in tests as well?
+13
I looked into io scheduler and wanted to switch to noop, but it’s not available in RHEL 8. I did not touch io-queue depth yet.
# cat /sys/block/sdf/queue/nr_requests
1013
Should be FIO parameter:
--iodepth=
+1
Well, depending on the test parameter I can reach 2,5GB/s. I mean… we probably will not be able to saturate this anyway in the near future. I just want to be sure that my RAID and fs setup correct as this is hard to modify once data is on the disk.
# fio --rw=write --name=test --size=50G --direct=1 --bs=512k --numjobs=20 --ioengine=libaio --iodepth=20 --refill_buffers --group_reporting
Run status group 0 (all jobs):
WRITE: bw=2463MiB/s (2582MB/s), 2463MiB/s-2463MiB/s (2582MB/s-2582MB/s), io=1000GiB (1074GB), run=415836-415836msec
Disk stats (read/write):
sdb: ios=0/2047239, merge=0/119, ticks=0/165897764, in_queue=164873511, util=100.00%
+1
Hm… this was during 5% CPU load.
# grep temperature /var/log/messages
May 12 19:07:11 sdeu2000 kernel: CPU46: Core temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU98: Core temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU45: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU97: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU98: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU46: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU26: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU78: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU94: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU27: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU85: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU88: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU42: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU91: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU28: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU80: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU33: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU41: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU36: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU93: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU39: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU44: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU51: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU82: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU32: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU87: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU84: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU79: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU103: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU35: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU48: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU96: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU30: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU43: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU81: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU38: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU29: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU50: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU34: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU100: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU95: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU102: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU40: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU92: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU47: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU99: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU90: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU49: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU89: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU101: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU86: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU31: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU37: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU83: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU99: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU81: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU98: Core temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU95: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU46: Core temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU102: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU94: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU89: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU87: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU35: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU29: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU43: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU42: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU32: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU49: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU38: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU90: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU41: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU93: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU44: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU83: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU31: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU91: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU39: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU82: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU30: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU80: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU28: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU51: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU84: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU100: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU48: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU27: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU79: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU36: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU88: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU47: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU45: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU78: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU97: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU37: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU40: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU92: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU26: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU103: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU101: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU50: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU86: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU85: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU33: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU34: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU98: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU46: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU96: Package temperature/speed normal
+13
Hm… this was during 5% CPU load.
# grep temperature /var/log/messages
May 12 19:07:11 sdeu2000 kernel: CPU46: Core temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU98: Core temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU45: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU97: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU98: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU46: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU26: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU78: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU94: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU27: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU85: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU88: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU42: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU91: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU28: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU80: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU33: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU41: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU36: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU93: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU39: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU44: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU51: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU82: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU32: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU87: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU84: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU79: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU103: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU35: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU48: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU96: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU30: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU43: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU81: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU38: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU29: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU50: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU34: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU100: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU95: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU102: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU40: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU92: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU47: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU99: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU90: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU49: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU89: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU101: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU86: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU31: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU37: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU83: Package temperature above threshold, cpu clock throttled (total events = 1)
May 12 19:07:11 sdeu2000 kernel: CPU99: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU81: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU98: Core temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU95: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU46: Core temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU102: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU94: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU89: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU87: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU35: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU29: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU43: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU42: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU32: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU49: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU38: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU90: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU41: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU93: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU44: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU83: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU31: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU91: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU39: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU82: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU30: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU80: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU28: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU51: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU84: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU100: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU48: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU27: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU79: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU36: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU88: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU47: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU45: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU78: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU97: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU37: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU40: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU92: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU26: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU103: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU101: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU50: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU86: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU85: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU33: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU34: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU98: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU46: Package temperature/speed normal
May 12 19:07:11 sdeu2000 kernel: CPU96: Package temperature/speed normal
What's your Workload-Profile respectively Power-Regulator settings?
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
