haswell BUG: unable to handle kernel paging request in rapl_scale

Found by

perf_fuzzer

Fixed in

3.19 98b008dff8452653909d9263efda925873e8d8bb

First Seen

3.19-rc5

Most recently Seen

3.19-rc5

Reproducible

yes

Type

crash

Found On

Haswell

Introduced By

89cbc76768c2fa4ed95545bf961f3a14ddfeed21 (3.18). Misplaced parenthesis in a patch when moving to __this_cpu_read.

Linux-kernel Mailing List Report

21 January 2015 -- perf: easily crash kernel with rapl event close

Kernel Splat

  1. Note, only able to trigger with paranoid set to 0. Was able to isolate down to two syscalls by taking a trace.
    Stopping after 30000
    Watchdog enabled with timeout 60s
    Will auto-exit if signal storm detected
    
    *** perf_fuzzer 0.29 *** by Vince Weaver
    
    	Linux version 3.19.0-rc5+ x86_64
    	Processor: Intel 6/60/3
    
    	Seeding random number generator with 1421689769
    	/proc/sys/kernel/perf_event_max_sample_rate currently: 100000/s
    	/proc/sys/kernel/perf_event_paranoid currently: 0
    	Logging perf_event_open() failures: no
    	Running fsync after every syscall: no
    	To reproduce, try: ./perf_fuzzer -s 30000 -r 1421689769
    
    Pid=2303, sleeping 1s
    ==================================================
    Fuzzing the following syscalls:
    	mmap perf_event_open close read write ioctl fork prctl poll 
    *NOT* Fuzzing the following syscalls:
    	
    Also attempting the following:
    	signal-handler-on-overflow busy-instruction-loop accessing-perf-proc-and-sys-files trashing-the-mmap-page 
    
    *NOT* attempting the following:
    
    ==================================================
    
    Iteration 10000
    	Open attempts: 150111  Successful: 914
    		EPERM : 24
    		ENOENT : 1106
    		E2BIG : 16198
    		EBADF : 7358
    		EINVAL : 123817
    		EOPNOTSUPP : 694
    		Type (Hardwar mc 1/3196)(#8 0/62)(#9 0/24)(#10 0/22)(#11 0/18)14 0/16893)
    	Cl4  Successful: 8
    	Read attempts: 837  Successful: 759
    	Write attempts: 829   Successful: 0
    	Ioctl attempts: 765  Successful: 351
    	Mmap attempts: 914  Successful: 400
    	Prctl attempts: 883  Successful: 883
    	Fork attempts: 458  Successful: 458
    	Poll attempts: 866  Successful: 726
    	Access attempts: 944  Successful: 477
    	Trash mmap attempts: 875  Successful: 875
    	Overflows: 0
    	SIGIOs due to RT signal queue	  full: 0
    
    [  784.067436] BUG: unable to handle kernel paging request at 000000000000cc68
    [  784.075656] IP: [] rapl_event_update+0x82/0xb0
    [  784.082789] PGD 0 
    [  784.085640] Oops: 0000 [#1] SMP 
    [  784.089821] Modules linked in: fuse x86_pkg_temp_thermal intel_powerclamp intel_rapl iosf_mbi coretemp kvm snd_hda_codec_realtek snd_hda_codec_hdmi snd_hda_codec_generic snd_hda_intel crct10dif_pclmul crc32_pclmul snd_hda_controller snd_hda_codec ghash_clmulni_intel snd_hwdep i915 aesni_intel snd_pcm aes_x86_64 snd_timer lrw iTCO_wdt iTCO_vendor_support evdev gf128mul snd glue_helper psmouse ppdev xhci_pci ablk_helper pcspkr serio_raw parport_pc soundcore cryptd xhci_hcd lpc_ich tpm_tis mei_me mei tpm i2c_i801 drm_kms_helper parport mfd_core battery video drm wmi i2c_algo_bit button processor sg sr_mod cdrom sd_mod ehci_pci ahci ehci_hcd libahci libata e1000e ptp scsi_mod crc32c_intel usbcore usb_common pps_core fan thermal thermal_sys
    [  784.164146] CPU: 5 PID: 0 Comm: swapper/5 Tainted: G        W      3.19.0-rc5+ #123
    [  784.173122] Hardware name: LENOVO 10AM000AUS/SHARKBAY, BIOS FBKT72AUS 01/26/2014
    [  784.181834] task: ffff880119470390 ti: ffff880119474000 task.ti: ffff880119474000
    [  784.190665] RIP: 0010:[]  [] rapl_event_update+0x82/0xb0
    [  784.200530] RSP: 0000:ffff88011eb43dd0  EFLAGS: 00010046
    [  784.207075] RAX: 0000000003f8077a RBX: ffff88011eb43ddc RCX: 0000000000000020
    [  784.215532] RDX: 0000000000000000 RSI: 0000000003f8077a RDI: 0000000000000000
    [  784.224001] RBP: ffff88011eb43e08 R08: 0000000000000000 R09: 0000000000000090
    [  784.232471] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000003f80574
    [  784.240952] R13: ffff8800ce2c81a0 R14: ffff8800ce2c8000 R15: 0000000003f8077a
    [  784.249414] FS:  0000000000000000(0000) GS:ffff88011eb40000(0000) knlGS:0000000000000000
    [  784.258911] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    [  784.265926] CR2: 000000000000cc68 CR3: 0000000001c13000 CR4: 00000000001407e0
    [  784.274389] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
    [  784.282869] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
    [  784.291350] Stack:
    [  784.294326]  ffff8800cda1cbe0 00000000cda1cb80 ffff8800ce2c8000 ffff8800cda1cb80
    [  784.303169]  0000000000000086 0000000000000004 ffffe8ffffd43f54 ffff88011eb43e38
    [  784.312047]  ffffffff81032db8 ffff8800ce2c8000 ffffe8ffffd43d10 000000b68e0eb5a0
    [  784.320889] Call Trace:
    [  784.324325]   
    [  784.326416]  [] rapl_pmu_event_stop+0x98/0x120
    [  784.334741]  [] rapl_pmu_event_del+0x13/0x20
    [  784.341796]  [] event_sched_out.isra.73+0xf6/0x240
    [  784.349420]  [] ? __perf_event_disable+0x58/0x140
    [  784.356927]  [] group_sched_out+0x4d/0xd0
    [  784.363716]  [] ? native_sched_clock+0x2a/0x90
    [  784.370964]  [] __perf_event_disable+0x106/0x140
    [  784.378366]  [] ? tick_nohz_irq_exit+0x29/0x30
    [  784.385560]  [] remote_function+0x50/0x60
    [  784.392245]  [] flush_smp_call_function_queue+0x62/0x140
    [  784.400323]  [] generic_smp_call_function_single_interrupt+0x13/0x60
    [  784.409466]  [] smp_trace_call_function_single_interrupt+0x38/0xc0
    [  784.418418]  [] trace_call_function_single_interrupt+0x6d/0x80
    [  784.426989]   
    [  784.429083]  [] ? cpuidle_enter_state+0x65/0x160
    [  784.437316]  [] ? cpuidle_enter_state+0x51/0x160
    [  784.444623]  [] cpuidle_enter+0x17/0x20
    [  784.451067]  [] cpu_startup_entry+0x311/0x3c0
    [  784.458060]  [] start_secondary+0x140/0x150
    [  784.464880] Code: 00 00 41 8b be 48 01 00 00 48 89 de e8 d8 42 02 00 66 90 49 89 c7 4c 89 e0 4d 0f b1 7d 00 4c 39 e0 75 d6 4c 89 f8 b9 20 00 00 00 <48> 8b 15 af a0 fd 7e 4c 29 e0 65 8b 52 38 48 98 29 d1 48 d3 e0 
    [  784.487063] RIP  [] rapl_event_update+0x82/0xb0
    [  784.494312]  RSP 
    [  784.498774] CR2: 000000000000cc68
    [  784.509591] ---[ end trace 667f9b8301b0c83d ]---
    [  784.515268] Kernel panic - not syncing: Fatal exception in interrupt
    [  784.522828] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
    [  784.534493] drm_kms_helper: panic occurred, switching back to text console
    [  784.542629] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
    

Back to perf_fuzzer bugs found