3.14-rc x86_schedule_events bug

Severity

System soft locks but requires reboot.

Found By

perf_fuzzer

Reproducible

No

First Seen

3.12.0

Most Recently Seen

3.14

Machines Affected

Seen on core2, haswell

Linux Kernel

10 March 2014: "another perf_fuzzer hard lockup", "perf_fuzzer lockup in x86_schedule_events()"

15 November 2013 -- Re: perf sw_event related lockup

14 November 2013 -- perf/x86 BUG/oops in __intel_shared_reg_get_constraints


  1. *** perf_fuzzer 0.28 *** by Vince Weaver                                        
                                                                                    
            Linux version 3.14.0 x86_64                                             
            Processor: Intel 6/60/3                                                 
                                                                                    
            Seeding random number generator with 1396367392                         
            /proc/sys/kernel/perf_event_max_sample_rate currently: 12500/s          
            /proc/sys/kernel/perf_event_paranoid currently: 1                       
            Logging perf_event_open() failures: no                                  
            Running fsync after every syscall: no                                   
            Run as: ./perf_fuzzer -t OCIRMQWPFpAi -s 50000                          
                                                                                    
    Pid=16033, sleeping 1s                                                          
    ==================================================                              
    Fuzzing the following syscalls:                                                 
            mmap perf_event_open close read write ioctl fork prctl poll             
    *NOT* Fuzzing the following syscalls:                                           
                                                                                    
    Also attempting the following:                                                  
            busy-instruction-loop accessing-perf-proc-and-sys-files trashing-the-mmap-page                                                                          
    *NOT* attempting the following:                                                 
            signal-handler-on-overflow                                              
    ==================================================                              
    Iteration 10000                                                                 
            Open attempts: 321168  Successful: 902                                  
                    EPERM : 22                                                      
                    ENOENT : 801                                                    
                    E2BIG : 37332                                                   
                    EBADF : 5433                                                    
                    EACCES : 203                                                    
                    EINVAL : 275868                                                 
                    ENOSPC : 19                                                     
                    EOPNOTSUPP : 588                                                
            Close attempts: 886  Successful: 886                                    
            Read attempts: 917  Successful: 800                                     
            Write attempts: 863  Successful: 0                                      
            Ioctl attempts: 906  Successful: 448                                    
            Mmap attempts: 902  Successful: 272                                     
            Prctl attempts: 943  Successful: 943                                    
            Fork attempts: 472  Successful: 472                                     
            Poll attempts: 875  Successful: 2                                       
            Access attempts: 869  Successful: 470                                   
            Trash mmap attempts: 874  Successful: 874                               
            Overflows: 0                                                            
            SIGIOs due to RT signal queue full: 0                                   
    [  854.517759] BUG: unable to handle kernel paging request at ffffffff8f30a928  
    [  854.525265] IP: [] x86_schedule_events+0x10c/0x270         
    [  854.532121] PGD 1a11067 PUD 1a12063 PMD 0                                    
    [  854.536590] Oops: 0000 [#1] SMP                                              
    [  854.540092] Dumping ftrace buffer:                                           
    [  854.543732]    (ftrace buffer empty)                                         
    [  854.547553] Modules linked in: nfsd auth_rpcgss oid_registry nfs_acl nfs lockd fscache sunrpc fuse x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi kvm snd_hda_intel crct10dif_pclmul snd_hda_codec snd_hwdep crc32_pclmul snd_pcm ghash_clmulni_intel snd_seq aesni_intel aes_x86_64 lrw gf128mul snd_seq_device glue_helper i915 drm_kms_helper ablk_helper snd_timer iTCO_wdt iTCO_vendor_support snd evdev cryptd drm lpc_ich parport_pc parport psmouse tpm_tis serio_raw tpm pcspkr mfd_core soundcore mei_me i2c_i801 i2c_algo_bit video processor i2c_core wmi mei button battery sg sd_mod sr_mod crc_t10dif crct10dif_common cdrom ahci ehci_pci xhci_hcd libahci ehci_hcd libata e1000e scsi_mod ptp crc32c_intel usbcore pps_core usb_common thermal fan thermal_sys
    [  854.625746] CPU: 0 PID: 16033 Comm: perf_fuzzer Not tainted 3.14.0 #16
    [  854.632736] Hardware name: LENOVO 10AM000AUS/SHARKBAY, BIOS FBKT72AUS 01/26/2014
    [  854.640649] task: ffff8801166bca50 ti: ffff8800cf95a000 task.ti: ffff8800cf95a000
    [  854.648720] RIP: 0010:[]  [] x86_schedule_events+0x10c/0x270
    [  854.658121] RSP: 0000:ffff88011ea03c30  EFLAGS: 00010046                     
    [  854.663815] RAX: 0000000000000000 RBX: ffff88011ea0cbe0 RCX: 00000000001004c9
    [  854.671450] RDX: ffffffff81a1e498 RSI: ffff88011715f000 RDI: 000000006b6b6b6b
    [  854.679109] RBP: ffff88011ea03c88 R08: 0000000000000000 R09: ffffffff81c33bc0
    [  854.686727] R10: 0048000000000000 R11: fffc0000ffdc0009 R12: ffff880118325800
    [  854.694373] R13: ffff88011ea0d120 R14: 0000000000000005 R15: 0000000000000000
    [  854.702041] FS:  00007f24c57bc700(0000) GS:ffff88011ea00000(0000) knlGS:0000000000000000
    [  854.710697] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033                
    [  854.716848] CR2: ffffffff8f30a928 CR3: 00000000cf080000 CR4: 00000000001407f0
    [  854.724448] DR0: 00000000013d2000 DR1: 00000000013d2000 DR2: 00000000013d2000
    [  854.732054] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 000000000f000628
    [  854.739666] Stack:                                                           
    [  854.741821]  ffff88011ea0d100 000000041ea03c70 0000000000000003 ffff88011ea03c98
    [  854.749818]  ffff88011ea0d120 0000000000000000 ffff88011ea0cbe0 ffff8801188c2e00
    [  854.757872]  0000000000000004 ffffffff81a1c340 0000000000000000 ffff88011ea03db8
    [  854.765864] Call Trace:                                                      
    [  854.768495]                                                             
    [  854.770575]  [] x86_pmu_commit_txn+0x49/0xa0               
    [  854.777033]  [] ? native_sched_clock+0x35/0x90             
    [  854.783482]  [] ? perf_event_update_userpage+0xd2/0x160    
    [  854.790742]  [] ? perf_event_update_userpage+0xea/0x160    
    [  854.797991]  [] ? event_sched_in.isra.77+0x148/0x1e0       
    [  854.804981]  [] group_sched_in+0x1c0/0x1e0                 
    [  854.811072]  [] ctx_sched_in+0x121/0x1e0                   
    [  854.816949]  [] perf_event_sched_in+0x60/0x90              
    [  854.823289]  [] perf_cpu_hrtimer_handler+0xe3/0x200        
    [  854.831517]  [] __run_hrtimer+0x86/0x1e0                   
    [  854.838638]  [] ? perf_event_context_sched_in+0xc0/0xc0    
    [  854.847169]  [] hrtimer_interrupt+0xf7/0x240               
    [  854.854733]  [] local_apic_timer_interrupt+0x37/0x60       
    [  854.862981]  [] smp_apic_timer_interrupt+0x3f/0x60         
    [  854.871031]  [] apic_timer_interrupt+0x6d/0x80             
    [  854.878670]                                                             
    [  854.880721] Code: 8d 40 01 74 32 48 8b b4 c3 28 05 00 00 48 83 c0 01 48 63 be 4c 01 00 00 4c 8b 8e 88 01 00 00 83 ff ff 0f 84 22 01 00 00 41 89 c0 <49> 0f a3 39 45 19 c9 45 85 c9 75 a0 45 31 e4 44 3b 45 b4 be 01                          
    [  854.906154] RIP  [] x86_schedule_events+0x10c/0x270
    [  854.914319]  RSP                                           
    [  854.919337] CR2: ffffffff8f30a928                                            
    [  855.157018] ---[ end trace 74a632e4f96aae48 ]---                             
    [  855.162968] Kernel panic - not syncing: Fatal exception in interrupt         
    [  855.170796] Dumping ftrace buffer:                                           
    [  855.175499]    (ftrace buffer empty)                                         
    [  855.180347] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)                                        
    
  2. *** perf_fuzzer 0.28 *** by Vince Weaver
    
            Linux version 3.14.0+ x86_64
            Processor: Intel 6/60/3
    
            Seeding random number generator with 1396285955
            /proc/sys/kernel/perf_event_max_sample_rate currently: 6250/s
            /proc/sys/kernel/perf_event_paranoid currently: 1
            Logging perf_event_open() failures: no
            Running fsync after every syscall: no
            Run as: ./perf_fuzzer -t OCIRMQWPFpAi -s 50000 
    
    Pid=28866, sleeping 1s
    ==================================================
    Fuzzing the following syscalls:
            mmap perf_event_open close read write ioctl fork prctl poll 
    *NOT* Fuzzing the following syscalls:
            
    Also attempting the following:
            busy-instruction-loop accessing-perf-proc-and-sys-files trashing-the-mmap-page 
    *NOT* attempting the following:
            signal-handler-on-overflow 
    ==================================================
    [ 3838.480346] BUG: unable to handle kernel paging request at ffffffff7d760be0
    [ 3838.491875] IP: [] x86_schedule_events+0x10c/0x270
    [ 3838.502767] PGD 1811067 PUD 0 
    [ 3838.510163] Oops: 0000 [#1] SMP 
    [ 3838.517638] Dumping ftrace buffer:
    [ 3838.525137]    (ftrace buffer empty)
    [ 3838.532700] Modules linked in: nfsd auth_rpcgss oid_registry nfs_acl nfs lockd fscache sunrpc fuse snd_hda_codec_hdmi i915 x86_pkg_temp_thermal intel_powerclamp coretemp drm_kms_helper drm kvm crct10dif_pclmul crc32_pclmul parport_pc ghash_clmulni_intel snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel aesni_intel snd_hda_codec snd_hwdep aes_x86_64 snd_pcm lrw gf128mul glue_helper tpm_tis iTCO_wdt iTCO_vendor_support parport tpm video battery ablk_helper lpc_ich snd_seq snd_seq_device i2c_algo_bit snd_timer psmouse button pcspkr i2c_i801 mei_me serio_raw wmi i2c_core processor mei cryptd snd mfd_core soundcore evdev sg sd_mod sr_mod crc_t10dif cdrom crct10dif_common hid_generic usbhid hid ahci libahci libata scsi_mod e1000e ehci_pci xhci_hcd ehci_hcd ptp pps_core crc32c_intel usbcore usb_common thermal fan thermal_sys
    [ 3838.636984] CPU: 2 PID: 28866 Comm: perf_fuzzer Tainted: G        W    3.14.0+ #14
    [ 3838.649070] Hardware name: LENOVO 10AM000AUS/SHARKBAY, BIOS FBKT72AUS 01/26/2014
    [ 3838.660899] task: ffff8801156e6450 ti: ffff8800cfe66000 task.ti: ffff8800cfe66000
    [ 3838.672858] RIP: 0010:[]  [] x86_schedule_events+0x10c/0x270
    [ 3838.686239] RSP: 0018:ffff8800cfe67950  EFLAGS: 00010046
    [ 3838.695912] RAX: 0000000000000000 RBX: ffff88011ea8ca60 RCX: 0000000000110000
    [ 3838.707555] RDX: ffffffff8181e128 RSI: ffff8801181bc400 RDI: ffffffffdead0000
    [ 3838.719257] RBP: ffff8800cfe679a8 R08: 0000000000000000 R09: ffffffff81a06be0
    [ 3838.730928] R10: ffff88011ea8ee00 R11: 0000000000000005 R12: ffff880116f5a400
    [ 3838.742611] R13: ffff88011ea8cf90 R14: 0000000000000004 R15: 0000000000000000
    [ 3838.754296] FS:  00007f8b3820c700(0000) GS:ffff88011ea80000(0000) knlGS:0000000000000000
    [ 3838.767010] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    [ 3838.777259] CR2: ffffffff7d760be0 CR3: 00000000cf246000 CR4: 00000000001407e0
    [ 3838.788988] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
    [ 3838.800750] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
    [ 3838.812414] Stack:
    [ 3838.818623]  ffff88011ea8cf80 000000021ea57068 0000000000000001 ffff8800cfe679b8
    [ 3838.830794]  ffff88011ea8cf90 0000000000000000 ffff88011ea8ca60 ffff880118768ac0
    [ 3838.842990]  0000000000000002 ffffffff8181afe0 0000000000000000 ffff8800cfe67ad8
    [ 3838.855151] Call Trace:
    [ 3838.861900]  [] x86_pmu_commit_txn+0x49/0xa0
    [ 3838.872274]  [] ? native_sched_clock+0x35/0x90
    [ 3838.882839]  [] ? arch_perf_update_userpage+0x7f/0x90
    [ 3838.894110]  [] ? perf_event_update_userpage+0xa6/0x120
    [ 3838.905494]  [] ? native_sched_clock+0x35/0x90
    [ 3838.915928]  [] ? sched_clock+0x9/0x10
    [ 3838.925549]  [] ? local_clock+0x25/0x30
    [ 3838.935186]  [] ? cpu_clock_event_stop+0x37/0x50
    [ 3838.945515]  [] ? perf_pmu_enable+0x27/0x30
    [ 3838.955307]  [] ? event_sched_out.isra.75+0x15b/0x1f0
    [ 3838.965936]  [] ? event_sched_in.isra.77+0x140/0x1d0
    [ 3838.976427]  [] group_sched_in+0x1c0/0x1e0
    [ 3838.985931]  [] ? ctx_sched_out+0x123/0x130
    [ 3838.995504]  [] ctx_sched_in+0x14b/0x1d0
    [ 3839.004775]  [] perf_event_sched_in+0x3a/0x90
    [ 3839.014481]  [] perf_event_context_sched_in+0x78/0xc0
    [ 3839.024927]  [] __perf_event_task_sched_in+0x18c/0x1a0
    [ 3839.035467]  [] finish_task_switch+0xa8/0x100
    [ 3839.045174]  [] __schedule+0x2cc/0x730
    [ 3839.054178]  [] schedule+0x29/0x70
    [ 3839.062864]  [] schedule_timeout+0x1b9/0x250
    [ 3839.072469]  [] ? debug_object_active_state+0x12b/0x170
    [ 3839.083157]  [] wait_for_completion+0x9f/0x110
    [ 3839.092953]  [] ? wake_up_state+0x20/0x20
    [ 3839.102279]  [] ? call_rcu_bh+0x20/0x20
    [ 3839.111418]  [] wait_rcu_gp+0x5a/0x70
    [ 3839.120362]  [] ? ftrace_raw_output_rcu_utilization+0x50/0x50
    [ 3839.131520]  [] synchronize_sched+0x3a/0x50
    [ 3839.141007]  [] perf_trace_event_unreg.isra.1+0x3b/0x90
    [ 3839.151643]  [] perf_trace_destroy+0x36/0x50
    [ 3839.161242]  [] tp_perf_event_destroy+0x9/0x10
    [ 3839.171085]  [] __free_event+0x23/0x60
    [ 3839.180047]  [] free_event+0x96/0x170
    [ 3839.188879]  [] SYSC_perf_event_open+0x51e/0xb40
    [ 3839.198778]  [] ? perf_event_enable+0x33/0xf0
    [ 3839.208402]  [] SyS_perf_event_open+0x9/0x10
    [ 3839.217942]  [] system_call_fastpath+0x1a/0x1f
    [ 3839.227649] Code: 8d 40 01 74 32 48 8b b4 c3 28 05 00 00 48 83 c0 01 48 63 be 4c 01 00 00 4c 8b 8e 88 01 00 00 83 ff ff 0f 84 22 01 00 00 41 89 c0 <49> 0f a3 39 45 19 c9 45 85 c9 75 a0 45 31 e4 44 3b 45 b4 be 01 
    [ 3839.256436] RIP  [] x86_schedule_events+0x10c/0x270
    [ 3839.266604]  RSP 
    [ 3839.273438] CR2: ffffffff7d760be0
    [ 3839.392197] ---[ end trace 15b4d5d0a680d664 ]---
    
  3. *** perf_fuzzer *** by Vince Weaver                                             
                                                                                    
    Seeding random number generator with 1394465165                                 
    Kernel max sample rate currently: 6250/s                                        
    ==================================================                              
    Fuzzing the following syscalls:                                                 
            mmap perf_event_open close read write ioctl fork prctl poll             
    *NOT* Fuzzing the following syscalls:                                           
                                                                                    
    Also attempting the following:                                                  
            signal-handler-on-overflow busy-instruction-loop accessing-perf-proc-and
    -sys-files trashing-the-mmap-page                                               
    *NOT* attempting the following:                                                 
                                                                                    
    ==================================================                              
    Iteration 10000                                                                 
            Open attempts: 346033  Successful: 922                                  
                    EPERM : 21                                                      
                    ENOENT : 845                                                    
                    E2BIG : 37400                                                   
                    EBADF : 5065                                                    
                    EINVAL : 301175                                                 
                    ENOSPC : 61                                                     
                    EOPNOTSUPP : 544                                                
            Close attempts: 914  Successful: 914                                    
            Read attempts: 913  Successful: 787                                     
            Write attempts: 849  Successful: 0                                      
            Ioctl attempts: 908  Successful: 323                                    
            Mmap attempts: 922  Successful: 219                                     
            Prctl attempts: 895  Successful: 895                                    
            Fork attempts: 455  Successful: 455                                     
            Poll attempts: 902  Successful: 4                                       
            Access attempts: 913  Successful: 463                                   
            Trash mmap attempts: 810  Successful: 810                               
            Overflows: 13                                                           
            SIGIOs due to RT signal queue full: 0                                   
    [ 7321.924284] BUG: unable to handle kernel NULL pointer dereference at         
      (null)                                                                        
    [ 7321.928004] IP: [] _raw_spin_lock_irqsave+0x11/0x2d        
    [ 7321.928004] PGD 118d2b067 PUD 11952d067 PMD 0                                
    [ 7321.928004] Oops: 0002 [#1] SMP                                              
    [ 7321.928004] Modules linked in: cpufreq_userspace cpufreq_stats cpufreq_powers
    ave cpufreq_conservative f71882fg mcs7830 usbnet evdev ohci_pci psmouse serio_ra
    w coretemp pcspkr ohci_hcd video wmi i2c_nforce2 acpi_cpufreq button processor t
    hermal_sys sg ehci_pci ehci_hcd sd_mod usbcore usb_common                       
    [ 7321.928004] CPU: 0 PID: 983 Comm: perf_fuzzer Not tainted 3.14.0-rc6 #36     
    [ 7321.928004] Hardware name: AOpen   DE7000/nMCP7ALPx-DE R1.06 Oct.19.2012, BIO
    S 080015  10/19/2012                                                            
    [ 7321.928004] task: ffff8801190b47e0 ti: ffff8800c7b28000 task.ti: ffff8800c7b2
    8000                                                                            
    [ 7321.928004] RIP: 0010:[]  [] _raw_spin_lo
    ck_irqsave+0x11/0x2d                                                            
    [ 7321.928004] RSP: 0018:ffff8800c7b29a08  EFLAGS: 00010046                     
    [ 7321.928004] RAX: 0000000000000046 RBX: ffff88011a592c00 RCX: 0000000000000000
    [ 7321.928004] RDX: 0000000000010000 RSI: ffff88011a592c00 RDI: 0000000000000000
    [ 7321.928004] RBP: ffff8800c7b29a08 R08: ffff88011a592c00 R09: 0000000000000000
    [ 7321.928004] R10: 0000000000000000 R11: ffff88011b280000 R12: ffff88011fc0b940
    [ 7321.928004] R13: 0000000000000000 R14: ffff88011a592d58 R15: 0000000000000000
    [ 7321.928004] FS:  0000000002195840(0063) GS:ffff88011fc00000(0000) knlGS:00000
    00000000000                                                                     
    [ 7321.928004] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b                
    [ 7321.928004] CR2: 0000000000000000 CR3: 0000000118dad000 CR4: 00000000000407f0
    [ 7321.928004] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
    [ 7321.928004] DR3: 0000000002197000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
    [ 7321.928004] Stack:                                                           
    [ 7321.928004]  ffff8800c7b29a58 ffffffff810198cc 0000000000000000 0000000000000
    000                                                                             
    [ 7321.928004]  ffff88011fff8d68 0000000000000000 ffff88011a592c00 ffff88011fc0b
    940                                                                             
    [ 7321.928004]  0000000000000040 0000000000000000 ffff8800c7b29a88 ffffffff81019
    a8f                                                                             
    [ 7321.928004] Call Trace:                                                      
    [ 7321.928004]  [] __intel_shared_reg_get_constraints+0x5e/0x1
    9c                                                                              
    [ 7321.928004]  [] intel_get_event_constraints+0x85/0x154     
    [ 7321.928004]  [] x86_schedule_events+0x54/0x185             
    [ 7321.928004]  [] x86_pmu_commit_txn+0x51/0x9a               
    [ 7321.928004]  [] ? __alloc_pages_nodemask+0x174/0x737       
    [ 7321.928004]  [] ? unmap_single_vma+0x605/0x669             
    [ 7321.928004]  [] ? event_sched_in+0x11d/0x12d               
    [ 7321.928004]  [] group_sched_in+0x98/0x147                  
    [ 7321.928004]  [] ctx_sched_in+0x15e/0x185                   
    [ 7321.928004]  [] perf_event_sched_in+0x69/0x72              
    [ 7321.928004]  [] __perf_install_in_context+0xc8/0xf1        
    [ 7321.928004]  [] ? perf_event_task_output+0x13d/0x13d       
    [ 7321.928004]  [] remote_function+0x3c/0x43                  
    [ 7321.928004]  [] smp_call_function_single+0x8a/0x101        
    [ 7321.928004]  [] task_function_call+0x44/0x4e               
    [ 7321.928004]  [] ? perf_event_sched_in+0x72/0x72            
    [ 7321.928004]  [] perf_install_in_context+0x81/0xbc          
    [ 7321.928004]  [] ? mutex_lock+0x16/0x37                     
    [ 7321.928004]  [] SYSC_perf_event_open+0x647/0x7f5           
    [ 7321.928004]  [] SyS_perf_event_open+0x9/0xb                
    [ 7321.928004]  [] system_call_fastpath+0x1a/0x1f             
    [ 7321.928004] Code: c1 07 89 c2 c1 ea 10 66 39 c2 74 0c 66 8b 07 66 39 d0 74 04
     f3 90 eb f4 c9 c3 55 48 89 e5 66 66 66 66 90 9c 58 fa ba 00 00 01 00  0f c1
     17 89 d1 c1 e9 10 66 39 d1 74 0c 66 8b 17 66 39 ca 74                          
    [ 7321.928004] RIP  [] _raw_spin_lock_irqsave+0x11/0x2d       
    [ 7321.928004]  RSP                                           
    [ 7321.928004] CR2: 0000000000000000                                            
    [ 7321.928004] ---[ end trace d36f8f2ae570dc66 ]---                             
    [ 7381.931999] INFO: rcu_sched detected stalls on CPUs/tasks: { 0} (detected by 
    1, t=15002 jiffies, g=55073, c=55072, q=25)                                     
    [ 7381.932000] sending NMI to all CPUs:                                         
    [ 7381.932000] NMI backtrace for cpu 1                                          
    [ 7381.932000] CPU: 1 PID: 1702 Comm: perf_fuzzer Tainted: G      D      3.14.0-
    rc6 #36                                                                         
    [ 7381.932000] Hardware name: AOpen   DE7000/nMCP7ALPx-DE R1.06 Oct.19.2012, BIO
    S 080015  10/19/2012                                                            
    [ 7381.932000] task: ffff880119785010 ti: ffff880119306000 task.ti: ffff88011930
    6000                                                                            
    [ 7381.932000] RIP: 0010:[]  [] __const_udel
    ay+0x1b/0x27                                                                    
    [ 7381.932000] RSP: 0000:ffff88011fc83d98  EFLAGS: 00000803                     
    [ 7381.932000] RAX: 0000000039558400 RBX: 0000000000000001 RCX: 0000000000000003
    [ 7381.932000] RDX: 00000000002658dc RSI: 0000000000000200 RDI: 0000000000418958
    [ 7381.932000] RBP: ffff88011fc83d98 R08: 0000000000000000 R09: 0000000000000008
    [ 7381.932000] R10: 0000000000000000 R11: 0000000000000005 R12: ffff88011fc8d510
    [ 7381.932000] R13: 0000000000000001 R14: ffffffff81a37300 R15: 0000000000000001
    [ 7381.932000] FS:  0000000002195840(0063) GS:ffff88011fc80000(0000) knlGS:00000
    00000000000                                                                     
    [ 7381.932000] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b                
    [ 7381.932000] CR2: 000000000040b040 CR3: 00000001196d2000 CR4: 00000000000407e0
    [ 7381.932000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000002197000
    [ 7381.932000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
    [ 7381.932000] Stack:                                                           
    [ 7381.932000]  ffff88011fc83db8 ffffffff8102c1ed 000000000000d720 ffffffff81a37
    300                                                                             
    [ 7381.932000]  ffff88011fc83e18 ffffffff81087ee3 0000000000000000 0000000000000
    019                                                                             
    [ 7381.932000]  ffffffff81a9eb60 0000000000000083 ffff88011fc83e08 ffff880119785
    010                                                                             
    [ 7381.932000] Call Trace:                                                      
    [ 7381.932000]                                                             
    [ 7381.932000]  [] arch_trigger_all_cpu_backtrace+0x79/0x8d   
    [ 7381.932000]  [] rcu_check_callbacks+0x462/0x5dd            
    [ 7381.932000]  [] update_process_times+0x41/0x6a             
    [ 7381.932000]  [] tick_sched_handle+0x4a/0x59                
    [ 7381.932000]  [] tick_sched_timer+0x5d/0x7c                 
    [ 7381.932000]  [] __run_hrtimer+0xba/0x145                   
    [ 7381.932000]  [] ? tick_nohz_handler+0xb0/0xb0              
    [ 7381.932000]  [] hrtimer_interrupt+0xd5/0x1c3               
    [ 7381.932000]  [] local_apic_timer_interrupt+0x58/0x5d       
    [ 7381.932000]  [] smp_trace_apic_timer_interrupt+0x53/0x91   
    [ 7381.932000]  [] trace_apic_timer_interrupt+0x6a/0x70       
    [ 7381.932000]                                                             
    [ 7381.932000] Code: 48 89 e5 66 66 66 66 90 ff 15 ee 8f 7b 00 c9 c3 55 48 8d 04
     bd 00 00 00 00 65 48 8b 14 25 e0 22 01 00 48 89 e5 48 6b d2 3e f7 e2 <48> 8d 7a
     01 ff 15 c7 8f 7b 00 c9 c3 55 48 89 e5 66 66 66 66 90                          
    [ 7322.969219] NMI backtrace for cpu 0                                          
    [ 7381.932000] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took t
    oo long to run: 692.154 msecs                                                   
    [ 7322.969219] CPU: 0 PID: 983 Comm: perf_fuzzer Tainted: G      D      3.14.0-r
    c6 #36                                                                          
    [ 7322.969219] Hardware name: AOpen   DE7000/nMCP7ALPx-DE R1.06 Oct.19.2012, BIO
    S 080015  10/19/2012                                                            
    [ 7322.969219] task: ffff8801190b47e0 ti: ffff8800c7b28000 task.ti: ffff8800c7b2
    8000                                                                            
    [ 7322.969219] RIP: 0010:[]  [] _raw_spin_lo
    ck+0x26/0x2a                                                                    
    [ 7322.969219] RSP: 0018:ffff88011fc03e48  EFLAGS: 00000097                     
    [ 7322.969219] RAX: 000000006f8f6f8e RBX: ffff88011a2a92c0 RCX: ffff88011fc03e98
    [ 7322.969219] RDX: 0000000000006f8f RSI: ffff88011a2a92c0 RDI: ffff88011fc15b2c
    [ 7322.969219] RBP: ffff88011fc03e48 R08: ffff88011fc0d010 R09: 0000000000000104
    [ 7322.969219] R10: ffff88011fc12ae0 R11: ffff88011fc0d010 R12: ffff88011a2a92c0
    [ 7322.969219] R13: ffff88011fc15c08 R14: 0000000000000000 R15: ffff88011fc0d0c8
    [ 7322.969219] FS:  0000000002195840(0063) GS:ffff88011fc00000(0000) knlGS:00000
    00000000000                                                                     
    [ 7322.969219] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b                
    [ 7322.969219] CR2: 0000000000000000 CR3: 0000000118dad000 CR4: 00000000000407f0
    [ 7322.969219] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
    [ 7322.969219] DR3: 0000000002197000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
    [ 7322.969219] Stack:                                                           
    [ 7322.969219]  ffff88011fc03e68 ffffffff810cb8df ffff88011fc03e98 ffff88011fc15
    b20                                                                             
    [ 7322.969219]  ffff88011fc03e98 ffffffff810cd5da ffff88011fc15c08 ffff88011fc0d
    010                                                                             
    [ 7322.969219]  ffff88011fc0d050 ffff88011fc03f28 ffff88011fc03ee8 ffffffff8105e
    5e4                                                                             
    [ 7322.969219] Call Trace:                                                      
    [ 7322.969219]                                                             
    [ 7322.969219]  [] perf_ctx_lock+0x15/0x27                    
    [ 7322.969219]  [] perf_cpu_hrtimer_handler+0x7a/0x1b0        
    [ 7322.969219]  [] __run_hrtimer+0xba/0x145                   
    [ 7322.969219]  [] ? __perf_install_in_context+0xf1/0xf1      
    [ 7322.969219]  [] hrtimer_interrupt+0xd5/0x1c3               
    [ 7322.969219]  [] local_apic_timer_interrupt+0x58/0x5d       
    [ 7322.969219]  [] smp_trace_apic_timer_interrupt+0x53/0x91   
    [ 7322.969219]  [] trace_apic_timer_interrupt+0x6a/0x70       
    [ 7322.969219]                                                             
    [ 7322.969219]  [] ? blocking_notifier_call_chain+0x14/0x16   
    [ 7322.969219]  [] ? acct_collect+0x175/0x17d                 
    [ 7322.969219]  [] do_exit+0x22f/0x947                        
    [ 7322.969219]  [] oops_end+0xb5/0xbd                         
    [ 7322.969219]  [] no_context+0x26d/0x27c                     
    [ 7322.969219]  [] __bad_area_nosemaphore+0x1bd/0x1dd         
    [ 7322.969219]  [] bad_area+0x47/0x4e                         
    [ 7322.969219]  [] __do_page_fault+0x25b/0x3ed                
    [ 7322.969219]  [] ? ctx_sched_in+0x15e/0x185                 
    [ 7322.969219]  [] ? perf_event_sched_in+0x69/0x72            
    [ 7322.969219]  [] trace_do_page_fault+0x8a/0x93              
    [ 7322.969219]  [] trace_page_fault+0x22/0x30                 
    [ 7322.969219]  [] ? _raw_spin_lock_irqsave+0x11/0x2d         
    [ 7322.969219]  [] __intel_shared_reg_get_constraints+0x5e/0x1
    9c                                                                              
    [ 7322.969219]  [] intel_get_event_constraints+0x85/0x154     
    [ 7322.969219]  [] x86_schedule_events+0x54/0x185             
    [ 7322.969219]  [] x86_pmu_commit_txn+0x51/0x9a               
    [ 7322.969219]  [] ? __alloc_pages_nodemask+0x174/0x737       
    [ 7322.969219]  [] ? unmap_single_vma+0x605/0x669             
    [ 7322.969219]  [] ? event_sched_in+0x11d/0x12d               
    [ 7322.969219]  [] group_sched_in+0x98/0x147                  
    [ 7322.969219]  [] ctx_sched_in+0x15e/0x185                   
    [ 7322.969219]  [] perf_event_sched_in+0x69/0x72              
    [ 7322.969219]  [] __perf_install_in_context+0xc8/0xf1        
    [ 7322.969219]  [] ? perf_event_task_output+0x13d/0x13d       
    [ 7322.969219]  [] remote_function+0x3c/0x43                  
    [ 7322.969219]  [] smp_call_function_single+0x8a/0x101        
    [ 7322.969219]  [] task_function_call+0x44/0x4e               
    [ 7322.969219]  [] ? perf_event_sched_in+0x72/0x72            
    [ 7322.969219]  [] perf_install_in_context+0x81/0xbc          
    [ 7322.969219]  [] ? mutex_lock+0x16/0x37                     
    [ 7322.969219]  [] SYSC_perf_event_open+0x647/0x7f5           
    [ 7322.969219]  [] SyS_perf_event_open+0x9/0xb                
    [ 7322.969219]  [] system_call_fastpath+0x1a/0x1f             
    [ 7322.969219] Code: 90 eb f4 c9 c3 55 48 89 e5 66 66 66 66 90 b8 00 00 01 00 f0
     0f c1 07 89 c2 c1 ea 10 66 39 c2 74 0c 66 8b 07 66 39 d0 74 04 f3 90  f4 c9
     c3 55 48 89 e5 41 56 41 55 41 54 53 66 66 66 66 90 48                          
    [ 7322.969219] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took t
    oo long to run: 1971.954 msecs                                                  
    
  4. Probably same bug
    [  573.054155] BUG: unable to handle kernel NULL pointer dereference at        )
    [  573.057564] IP: [] _raw_spin_lock_irqsave+0x11/0x2d
    [  573.057564] PGD c814b067 PUD cb183067 PMD 0 
    [  573.057564] Oops: 0002 [#1] SMP 
    [  573.057564] Modules linked in: cpufreq_userspace cpufreq_stats cpufreq_powern
    [  573.057564] CPU: 1 PID: 3926 Comm: perf_fuzzer Tainted: G        W    3.12.04
    [  573.057564] Hardware name: AOpen   DE7000/nMCP7ALPx-DE R1.06 Oct.19.2012, BI2
    [  573.057564] task: ffff880037cfd0c0 ti: ffff8800ca4ae000 task.ti: ffff8800ca40
    [  573.057564] RIP: 0010:[]  [] _raw_spin_ld
    [  573.057564] RSP: 0018:ffff8800ca4afa08  EFLAGS: 00010046
    [  573.057564] RAX: 0000000000000046 RBX: ffff8800c813e800 RCX: 0000000000000000
    [  573.057564] RDX: 0000000000010000 RSI: ffff8800c813e800 RDI: 0000000000000000
    [  573.057564] RBP: ffff8800ca4afa08 R08: ffff8800c813e800 R09: ffff8800ca4afdf8
    [  573.057564] R10: ffff88011a2261c0 R11: 0000000000000246 R12: ffff88011fc8b940
    [  573.057564] R13: 0000000000000000 R14: ffff8800c813e948 R15: 0000000000000000
    [  573.057564] FS:  00007fc238e92700(0000) GS:ffff88011fc80000(0000) knlGS:00000
    [  573.057564] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
    [  573.057564] CR2: 0000000000000000 CR3: 00000000c88c4000 CR4: 00000000000407e0
    [  573.057564] DR0: 0000000000033c20 DR1: 0000000000000000 DR2: 0000000000000000
    [  573.057564] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
    [  573.057564] Stack:
    [  573.057564]  ffff8800ca4afa58 ffffffff810190b0 0000000000000000 0000000000000
    [  573.057564]  ffff88011fff8680 0000000000000000 ffff8800c813e800 ffff88011fc80
    [  573.057564]  0000000000000040 0000000000000000 ffff8800ca4afa88 ffffffff81013
    [  573.057564] Call Trace:
    [  573.057564]  [] __intel_shared_reg_get_constraints+0x5e/0xc
    [  573.057564]  [] intel_get_event_constraints+0x85/0x154
    [  573.057564]  [] x86_schedule_events+0x54/0x185
    [  573.057564]  [] ? x86_pmu_add+0xf8/0x110
    [  573.057564]  [] x86_pmu_commit_txn+0x51/0x9a
    [  573.057564]  [] ? __alloc_pages_nodemask+0x173/0x718
    [  573.057564]  [] ? event_sched_in+0x7a/0x109
    [  573.057564]  [] group_sched_in+0x98/0x147
    [  573.057564]  [] ? sched_clock_cpu+0xc3/0xd1
    [  573.057564]  [] ctx_sched_in+0x15e/0x185
    [  573.057564]  [] perf_event_sched_in+0x69/0x72
    [  573.057564]  [] __perf_install_in_context+0xc8/0xf1
    [  573.057564]  [] ? perf_event_task_output+0x13d/0x13d
    [  573.057564]  [] remote_function+0x3c/0x43
    [  573.057564]  [] smp_call_function_single+0x8a/0x101
    [  573.057564]  [] task_function_call+0x44/0x4e
    [  573.057564]  [] ? perf_event_sched_in+0x72/0x72
    [  573.057564]  [] perf_install_in_context+0x81/0xbc
    [  573.057564]  [] ? mutex_lock+0x16/0x37
    [  573.057564]  [] SYSC_perf_event_open+0x634/0x7e9
    [  573.057564]  [] SyS_perf_event_open+0x9/0xb
    [  573.057564]  [] system_call_fastpath+0x1a/0x1f
    [  573.057564] Code: c1 07 89 c2 c1 ea 10 66 39 c2 74 0c 66 8b 07 66 39 d0 74 0 
    [  573.057564] RIP  [] _raw_spin_lock_irqsave+0x11/0x2d
    [  573.057564]  RSP 
    [  573.057564] CR2: 0000000000000000
    [  573.057564] ---[ end trace ee0dc4a441b301c0 ]---
    [  577.000000] ------------[ cut here ]------------
    [  577.000000] WARNING: CPU: 0 PID: 15546 at kernel/watchdog.c:245 watchdog_ove)
    [  577.000000] Watchdog detected hard LOCKUP on cpu 0
    [  577.000000] Modules linked in: cpufreq_userspace cpufreq_stats cpufreq_powern
    [  577.000000] CPU: 0 PID: 15546 Comm: perf_fuzzer Tainted: G      D W    3.12.4
    [  577.000000] Hardware name: AOpen   DE7000/nMCP7ALPx-DE R1.06 Oct.19.2012, BI2
    [  577.000000]  00000000000000f5 ffff88011fc07bc8 ffffffff8151d8ec 0000000000005
    [  577.000000]  ffff88011fc07c18 ffff88011fc07c08 ffffffff8103cda9 ffff88011fc08
    [  577.000000]  ffffffff810a137f ffff88011b313400 0000000000000000 ffff88011fc08
    [  577.000000] Call Trace:
    [  577.000000]    [] dump_stack+0x49/0x5d
    [  577.000000]  [] warn_slowpath_common+0x81/0x9b
    [  577.000000]  [] ? watchdog_overflow_callback+0x9b/0xa6
    [  577.000000]  [] warn_slowpath_fmt+0x46/0x48
    [  577.000000]  [] watchdog_overflow_callback+0x9b/0xa6
    [  577.000000]  [] __perf_event_overflow+0x137/0x1c1
    [  577.000000]  [] ? intel_pmu_enable_all+0x4c/0xa4
    [  577.000000]  [] perf_event_overflow+0x14/0x16
    [  577.000000]  [] intel_pmu_handle_irq+0x2b8/0x34d
    [  577.000000]  [] ? __schedule+0x514/0x54b
    [  577.000000]  [] perf_event_nmi_handler+0x2d/0x4a
    [  577.000000]  [] nmi_handle+0x5e/0x13a
    [  577.000000]  [] do_nmi+0xa8/0x2c0
    [  577.000000]  [] end_repeat_nmi+0x1e/0x2e
    [  577.000000]  [] ? _raw_spin_lock+0x26/0x2a
    [  577.000000]  [] ? _raw_spin_lock+0x26/0x2a
    [  577.000000]  [] ? _raw_spin_lock+0x26/0x2a
    [  577.000000]  <>    [] sched_rt_period_timer+0x8be
    [  577.000000]  [] __run_hrtimer+0xba/0x145
    [  577.000000]  [] ? init_rt_bandwidth+0x46/0x46
    [  577.000000]  [] hrtimer_interrupt+0xd5/0x1c3
    [  577.000000]  [] local_apic_timer_interrupt+0x58/0x5d
    [  577.000000]  [] smp_apic_timer_interrupt+0x2a/0x3c
    [  577.000000]  [] apic_timer_interrupt+0x6a/0x70
    [  577.000000]   
    [  577.000000] ---[ end trace ee0dc4a441b301c1 ]---
    
  5. [ 2715.629603] BUG: unable to handle kernel NULL pointer dereference at           (null)
    [ 2715.632001] IP: [] _raw_spin_lock_irqsave+0x11/0x2d
    [ 2715.632001] PGD bc11f067 PUD c234d067 PMD 0 
    [ 2715.632001] Oops: 0002 [#1] SMP 
    [ 2715.632001] Modules linked in: cpufreq_userspace cpufreq_stats cpufreq_powersave cpufreq_conservative f71882fg mcs7830 acpi_cpufreq usbnet psmouse evdev serio_raw coretemp pcspkr button wmi video ohci_pci ohci_hcd processor i2c_nforce2 thermal_sys ehci_pci sg ehci_hcd sd_mod usbcore usb_common
    [ 2715.632001] CPU: 1 PID: 9731 Comm: perf_fuzzer Tainted: G        W    3.12.0 #3
    [ 2715.632001] Hardware name: AOpen   DE7000/nMCP7ALPx-DE R1.06 Oct.19.2012, BIOS 080015  10/19/2012
    [ 2715.632001] task: ffff8800cbad9890 ti: ffff8800bc0c2000 task.ti: ffff8800bc0c2000
    [ 2715.632001] RIP: 0010:[]  [] _raw_spin_lock_irqsave+0x11/0x2d
    [ 2715.632001] RSP: 0018:ffff8800bc0c3af8  EFLAGS: 00010046
    [ 2715.632001] RAX: 0000000000000046 RBX: ffff8800bec9c000 RCX: 0000000000000000
    [ 2715.632001] RDX: 0000000000010000 RSI: ffff8800bec9c000 RDI: 0000000000000000
    [ 2715.632001] RBP: ffff8800bc0c3af8 R08: ffff8800bec9c000 R09: ffff8800bc0c3d18
    [ 2715.632001] R10: 00007f7bf54c80a4 R11: 0000000000000246 R12: ffff88011fc8b940
    [ 2715.632001] R13: 0000000000000000 R14: ffff8800bec9c148 R15: 0000000000000000
    [ 2715.632001] FS:  00007f7bf56e0700(0000) GS:ffff88011fc80000(0000) knlGS:0000000000000000
    [ 2715.632001] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    [ 2715.632001] CR2: 0000000000000000 CR3: 00000000bcbc6000 CR4: 00000000000407e0
    [ 2715.632001] DR0: 00000000a86fb8ff DR1: 000000000080ffcd DR2: 000040dbdfdbdbdb
    [ 2715.632001] DR3: 0000000100000000 DR6: 00000000ffff0ff0 DR7: 0000000000030602
    [ 2715.632001] Stack:
    [ 2715.632001]  ffff8800bc0c3b48 ffffffff81018db4 0000000000000000 0000000000000000
    [ 2715.632001]  00007fffb0ae0fff 0000000000000000 ffff8800bec9c000 ffff88011fc8b940
    [ 2715.632001]  0000000000000040 0000000000000000 ffff8800bc0c3b78 ffffffff81018f77
    [ 2715.632001] Call Trace:
    [ 2715.632001]  [] ? __intel_shared_reg_get_constraints+0x5e/0x19c
    [ 2715.632001]  [] ? intel_get_event_constraints+0x85/0x154
    [ 2715.632001]  [] ? x86_schedule_events+0x53/0x17f
    [ 2715.632001]  [] ? x86_pmu_commit_txn+0x4a/0x8d
    [ 2715.632001]  [] ? perf_output_copy+0x2f/0x85
    [ 2715.632001]  [] ? perf_swevent_add+0x37/0xf7
    [ 2715.632001]  [] ? event_sched_in+0x79/0x107
    [ 2715.632001]  [] ? group_sched_in+0x8f/0x142
    [ 2715.632001]  [] ? event_sched_in+0x79/0x107
    [ 2715.632001]  [] ? group_sched_in+0x8f/0x142
    [ 2715.632001]  [] ? __perf_event_enable+0x118/0x173
    [ 2715.632001]  [] ? perf_event_task_output+0x117/0x117
    [ 2715.632001]  [] ? remote_function+0x34/0x39
    [ 2715.632001]  [] ? smp_call_function_single+0x8a/0x101
    [ 2715.632001]  [] ? task_function_call+0x42/0x4c
    [ 2715.632001]  [] ? __perf_event_task_sched_in+0x108/0x108
    [ 2715.632001]  [] ? perf_event_enable+0x10a/0x141
    [ 2715.632001]  [] ? mutex_lock+0x16/0x37
    [ 2715.632001]  [] ? perf_install_in_context+0xb8/0xb8
    [ 2715.632001]  [] ? perf_event_for_each_child+0x56/0x88
    [ 2715.632001]  [] ? perf_event_task_enable+0x3f/0x6c
    [ 2715.632001]  [] ? SyS_prctl+0x151/0x341
    [ 2715.632001]  [] ? system_call_fastpath+0x1a/0x1f
    [ 2715.632001] Code: c1 07 89 c2 c1 ea 10 66 39 c2 74 0c 66 8b 07 66 39 d0 74 04 f3 90 eb f4 c9 c3 55 48 89 e5 66 66 66 66 90 9c 58 fa ba 00 00 01 00  0f c1 17 89 d1 c1 e9 10 66 39 d1 74 0c 66 8b 17 66 39 ca 74 
    [ 2715.632001] RIP  [] _raw_spin_lock_irqsave+0x11/0x2d
    [ 2715.632001]  RSP 
    [ 2715.632001] CR2: 0000000000000000
    [ 2715.632001] ---[ end trace 2f66a1bf606d6b56 ]---
    
    

Back to bugs found page