[Bug 577785] Re: Lost IRQ drop harddisk

alf at all.de alf at all.de
Wed May 19 13:37:18 UTC 2010


And now (still 2.6.34-999) when the raids are syncing the initial
behavier occur (Lost IRQ, dropping drives) ...

[ 2085.811299] CE: hpet increased min_delta_ns to 7500 nsec
[ 8050.011293] ata1: lost interrupt (Status 0x50)
[ 8050.012395] sd 0:0:0:0: [sda] Unhandled error code
[ 8050.013751] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8050.015121] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 5e 87 1b 00 00 08 00
[ 8050.016592] end_request: I/O error, dev sda, sector 6194971
[ 8050.017704] Buffer I/O error on device dm-1, logical block 750218
[ 8050.018609] lost page write due to I/O error on dm-1
[ 8080.040103] ata1: lost interrupt (Status 0x50)
[ 8080.040995] sd 0:0:0:0: [sda] Unhandled error code
[ 8080.041858] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8080.042828] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 5e 87 c3 00 00 08 00
[ 8080.043738] end_request: I/O error, dev sda, sector 6195139
[ 8080.044813] Buffer I/O error on device dm-1, logical block 750239
[ 8080.046276] lost page write due to I/O error on dm-1
[ 8111.043832] ata1: lost interrupt (Status 0x50)
[ 8111.044933] sd 0:0:0:0: [sda] Unhandled error code
[ 8111.046353] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8111.048085] sd 0:0:0:0: [sda] CDB: Read(10): 28 00 00 5c 0f 13 00 00 08 00
[ 8111.049970] end_request: I/O error, dev sda, sector 6033171
[ 8142.010949] ata1: lost interrupt (Status 0x50)
[ 8142.012058] sd 0:0:0:0: [sda] Unhandled error code
[ 8142.013166] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8142.014373] sd 0:0:0:0: [sda] CDB: Read(10): 28 00 00 5c 0f 13 00 00 08 00
[ 8142.015971] end_request: I/O error, dev sda, sector 6033171
[ 8161.060308] INFO: task jbd2/dm-1-8:615 blocked for more than 120 seconds.
[ 8161.061450] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8161.062595] jbd2/dm-1-8   D ffff880001e95780     0   615      2 0x00000000
[ 8161.063675]  ffff880221d93d10 0000000000000046 0000000000000000 ffff880221d93fd8
[ 8161.064863]  ffff880223f11700 0000000000015780 0000000000015780 ffff880221d93fd8
[ 8161.066282]  0000000000015780 ffff880221d93fd8 0000000000015780 ffff880223f11700
[ 8161.067496] Call Trace:
[ 8161.068891]  [<ffffffff812138e7>] jbd2_journal_commit_transaction+0x1e7/0x11d0
[ 8161.070346]  [<ffffffff8100985b>] ? __switch_to+0xbb/0x2e0
[ 8161.071493]  [<ffffffff8105118e>] ? put_prev_entity+0x2e/0x80
[ 8161.072598]  [<ffffffff81540044>] ? _raw_spin_lock_irqsave+0x34/0x50
[ 8161.073878]  [<ffffffff8106eb2b>] ? lock_timer_base+0x3b/0x70
[ 8161.073882]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8161.073886]  [<ffffffff81219693>] kjournald2+0xc3/0x230
[ 8161.073889]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8161.073891]  [<ffffffff812195d0>] ? kjournald2+0x0/0x230
[ 8161.073894]  [<ffffffff8107e9d6>] kthread+0x96/0xa0
[ 8161.073897]  [<ffffffff8100be64>] kernel_thread_helper+0x4/0x10
[ 8161.073900]  [<ffffffff8107e940>] ? kthread+0x0/0xa0
[ 8161.073902]  [<ffffffff8100be60>] ? kernel_thread_helper+0x0/0x10
[ 8161.073918] INFO: task kjournald:1695 blocked for more than 120 seconds.
[ 8161.073920] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8161.073921] kjournald     D ffff880001e15780     0  1695      2 0x00000000
[ 8161.073924]  ffff880225ef7d40 0000000000000046 ffff880225ef7cf0 ffff880225ef7fd8
[ 8161.073927]  ffff8802252e5c00 0000000000015780 0000000000015780 ffff880225ef7fd8
[ 8161.073929]  0000000000015780 ffff880225ef7fd8 0000000000015780 ffff8802252e5c00
[ 8161.073931] Call Trace:
[ 8161.073935]  [<ffffffff8120bd0a>] journal_commit_transaction+0x17a/0xdf0
[ 8161.073938]  [<ffffffff8100985b>] ? __switch_to+0xbb/0x2e0
[ 8161.073941]  [<ffffffff8105118e>] ? put_prev_entity+0x2e/0x80
[ 8161.073948]  [<ffffffff81033d39>] ? default_spin_lock_flags+0x9/0x10
[ 8161.073950]  [<ffffffff81540044>] ? _raw_spin_lock_irqsave+0x34/0x50
[ 8161.073953]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8161.073956]  [<ffffffff8106f971>] ? try_to_del_timer_sync+0x51/0xe0
[ 8161.073958]  [<ffffffff81540044>] ? _raw_spin_lock_irqsave+0x34/0x50
[ 8161.073961]  [<ffffffff8120f8d3>] kjournald+0xf3/0x260
[ 8161.073964]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8161.073966]  [<ffffffff8120f7e0>] ? kjournald+0x0/0x260
[ 8161.073969]  [<ffffffff8107e9d6>] kthread+0x96/0xa0
[ 8161.073971]  [<ffffffff8100be64>] kernel_thread_helper+0x4/0x10
[ 8161.073974]  [<ffffffff8107e940>] ? kthread+0x0/0xa0
[ 8161.073976]  [<ffffffff8100be60>] ? kernel_thread_helper+0x0/0x10
[ 8161.073980] INFO: task rsyslogd:1884 blocked for more than 120 seconds.
[ 8161.073982] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8161.073983] rsyslogd      D ffff880001f95780     0  1884      1 0x00000000
[ 8161.073985]  ffff88021fe117a8 0000000000000086 ffff880215d27400 ffff88021fe11fd8
[ 8161.073988]  ffff880225c2c500 0000000000015780 0000000000015780 ffff88021fe11fd8
[ 8161.073990]  0000000000015780 ffff88021fe11fd8 0000000000015780 ffff880225c2c500
[ 8161.073992] Call Trace:
[ 8161.073995]  [<ffffffff8153e007>] io_schedule+0x47/0x70
[ 8161.073999]  [<ffffffff81169435>] sync_buffer+0x45/0x50
[ 8161.074001]  [<ffffffff8153e70a>] __wait_on_bit_lock+0x5a/0xb0
[ 8161.074003]  [<ffffffff811693f0>] ? sync_buffer+0x0/0x50
[ 8161.074006]  [<ffffffff811693f0>] ? sync_buffer+0x0/0x50
[ 8161.074008]  [<ffffffff8153e7dd>] out_of_line_wait_on_bit_lock+0x7d/0x90
[ 8161.074011]  [<ffffffff8107edf0>] ? wake_bit_function+0x0/0x40
[ 8161.074016]  [<ffffffff81168f7c>] __lock_buffer+0x2c/0x30
[ 8161.074018]  [<ffffffff8120b48a>] do_get_write_access+0x48a/0x4b0
[ 8161.074021]  [<ffffffff8120b631>] journal_get_write_access+0x31/0x50
[ 8161.074027]  [<ffffffff811c1754>] __ext3_journal_get_write_access+0x34/0x70
[ 8161.074030]  [<ffffffff811b400a>] ext3_free_data+0x3a/0x130
[ 8161.074033]  [<ffffffff811b1ea2>] ? ext3_get_branch+0xb2/0x110
[ 8161.074035]  [<ffffffff811b42c7>] ext3_free_branches+0x1c7/0x1d0
[ 8161.074038]  [<ffffffff811b535f>] ext3_truncate+0x34f/0x540
[ 8161.074041]  [<ffffffff811b6d54>] ? ext3_orphan_add+0x1c4/0x1d0
[ 8161.074043]  [<ffffffff811b57a9>] ext3_truncate_failed_write+0x29/0x30
[ 8161.074046]  [<ffffffff811b5d15>] ext3_write_begin+0x1c5/0x210
[ 8161.074048]  [<ffffffff811b4b10>] ? ext3_get_block+0x0/0x120
[ 8161.074052]  [<ffffffff810f2b51>] generic_perform_write+0xc1/0x1a0
[ 8161.074055]  [<ffffffff8116343a>] ? __mark_inode_dirty+0x3a/0x190
[ 8161.074058]  [<ffffffff810f2c95>] generic_file_buffered_write+0x65/0xa0
[ 8161.074060]  [<ffffffff810f3e01>] __generic_file_aio_write+0x251/0x3f0
[ 8161.074063]  [<ffffffff811b324d>] ? ext3_mark_iloc_dirty+0x1d/0x30
[ 8161.074066]  [<ffffffff811bc434>] ? __ext3_journal_stop+0x34/0x60
[ 8161.074068]  [<ffffffff810f3ffc>] generic_file_aio_write+0x5c/0xb0
[ 8161.074073]  [<ffffffff81141525>] do_sync_write+0xd5/0x120
[ 8161.074079]  [<ffffffff8115112a>] ? vfs_ioctl+0x3a/0xc0
[ 8161.074084]  [<ffffffff81249676>] ? security_file_permission+0x16/0x20
[ 8161.074086]  [<ffffffff811426fc>] vfs_write+0xcc/0x1a0
[ 8161.074089]  [<ffffffff811428c5>] sys_write+0x55/0x90
[ 8161.074091]  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
[ 8161.074098] INFO: task flush-252:1:2196 blocked for more than 120 seconds.
[ 8161.074099] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8161.074100] flush-252:1   D ffff880001e95780     0  2196      2 0x00000000
[ 8161.074103]  ffff8802247c16b0 0000000000000046 ffff88021fef4d20 ffff8802247c1fd8
[ 8161.074105]  ffff88022462ae00 0000000000015780 0000000000015780 ffff8802247c1fd8
[ 8161.074107]  0000000000015780 ffff8802247c1fd8 0000000000015780 ffff88022462ae00
[ 8161.074110] Call Trace:
[ 8161.074112]  [<ffffffff8153e007>] io_schedule+0x47/0x70
[ 8161.074117]  [<ffffffff8128abe0>] get_request_wait+0xd0/0x170
[ 8161.074121]  [<ffffffff812997d5>] ? cfq_find_rq_fmerge+0x65/0x80
[ 8161.074124]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8161.074127]  [<ffffffff81284053>] ? elv_merge+0xf3/0x110
[ 8161.074130]  [<ffffffff8128ad0f>] __make_request+0x8f/0x480
[ 8161.074132]  [<ffffffff8153f306>] ? down_read+0x16/0x30
[ 8161.074135]  [<ffffffff8128977c>] generic_make_request+0x1dc/0x3f0
[ 8161.074137]  [<ffffffff810f41fd>] ? mempool_alloc+0x6d/0x140
[ 8161.074140]  [<ffffffff8116bac6>] ? block_write_full_page_endio+0x116/0x120
[ 8161.074144]  [<ffffffff811da620>] ? noalloc_get_block_write+0x0/0x60
[ 8161.074146]  [<ffffffff81289a0a>] submit_bio+0x7a/0x100
[ 8161.074149]  [<ffffffff811686d1>] submit_bh+0x101/0x150
[ 8161.074152]  [<ffffffff8116b835>] __block_write_full_page+0x1c5/0x340
[ 8161.074154]  [<ffffffff81169500>] ? end_buffer_async_write+0x0/0x160
[ 8161.074156]  [<ffffffff811da620>] ? noalloc_get_block_write+0x0/0x60
[ 8161.074159]  [<ffffffff81169500>] ? end_buffer_async_write+0x0/0x160
[ 8161.074161]  [<ffffffff8116bac6>] block_write_full_page_endio+0x116/0x120
[ 8161.074164]  [<ffffffff811da620>] ? noalloc_get_block_write+0x0/0x60
[ 8161.074166]  [<ffffffff8116bae5>] block_write_full_page+0x15/0x20
[ 8161.074169]  [<ffffffff811d6f2d>] ext4_writepage+0x1bd/0x290
[ 8161.074171]  [<ffffffff811d74db>] mpage_da_submit_io+0x13b/0x1a0
[ 8161.074174]  [<ffffffff811daf65>] ext4_da_writepages+0x3e5/0x590
[ 8161.074177]  [<ffffffff810fa140>] ? __writepage+0x0/0x40
[ 8161.074180]  [<ffffffff8107ec77>] ? bit_waitqueue+0x17/0xd0
[ 8161.074183]  [<ffffffff810fb6a1>] do_writepages+0x21/0x40
[ 8161.074185]  [<ffffffff811624bb>] writeback_single_inode+0xdb/0x250
[ 8161.074188]  [<ffffffff8116282d>] writeback_sb_inodes+0xdd/0x160
[ 8161.074191]  [<ffffffff81162d42>] writeback_inodes_wb+0xd2/0x110
[ 8161.074193]  [<ffffffff81162ed5>] wb_writeback+0x155/0x210
[ 8161.074195]  [<ffffffff81540044>] ? _raw_spin_lock_irqsave+0x34/0x50
[ 8161.074198]  [<ffffffff81163035>] wb_check_old_data_flush+0xa5/0xb0
[ 8161.074201]  [<ffffffff8106eb60>] ? process_timeout+0x0/0x10
[ 8161.074203]  [<ffffffff811631ea>] wb_do_writeback+0xaa/0xc0
[ 8161.074206]  [<ffffffff81163243>] bdi_writeback_task+0x43/0xc0
[ 8161.074209]  [<ffffffff8110afe0>] ? bdi_start_fn+0x0/0x100
[ 8161.074211]  [<ffffffff8110b06c>] bdi_start_fn+0x8c/0x100
[ 8161.074214]  [<ffffffff8110afe0>] ? bdi_start_fn+0x0/0x100
[ 8161.074216]  [<ffffffff8107e9d6>] kthread+0x96/0xa0
[ 8161.074219]  [<ffffffff8100be64>] kernel_thread_helper+0x4/0x10
[ 8161.074221]  [<ffffffff8107e940>] ? kthread+0x0/0xa0
[ 8161.074224]  [<ffffffff8100be60>] ? kernel_thread_helper+0x0/0x10
[ 8161.074226] INFO: task flush-252:3:2197 blocked for more than 120 seconds.
[ 8161.074227] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8161.074228] flush-252:3   D ffff880001e95780     0  2197      2 0x00000000
[ 8161.074231]  ffff8802259af790 0000000000000046 ffff88021fef4d20 ffff8802259affd8
[ 8161.074233]  ffff880224629700 0000000000015780 0000000000015780 ffff8802259affd8
[ 8161.074235]  0000000000015780 ffff8802259affd8 0000000000015780 ffff880224629700
[ 8161.074237] Call Trace:
[ 8161.074239]  [<ffffffff8153e007>] io_schedule+0x47/0x70
[ 8161.074242]  [<ffffffff8128abe0>] get_request_wait+0xd0/0x170
[ 8161.074245]  [<ffffffff812997d5>] ? cfq_find_rq_fmerge+0x65/0x80
[ 8161.074248]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8161.074250]  [<ffffffff81284053>] ? elv_merge+0xf3/0x110
[ 8161.074252]  [<ffffffff8128ad0f>] __make_request+0x8f/0x480
[ 8161.074255]  [<ffffffff8153f306>] ? down_read+0x16/0x30
[ 8161.074257]  [<ffffffff8128977c>] generic_make_request+0x1dc/0x3f0
[ 8161.074260]  [<ffffffff810f41fd>] ? mempool_alloc+0x6d/0x140
[ 8161.074262]  [<ffffffff81289a0a>] submit_bio+0x7a/0x100
[ 8161.074265]  [<ffffffff811686d1>] submit_bh+0x101/0x150
[ 8161.074267]  [<ffffffff8116b835>] __block_write_full_page+0x1c5/0x340
[ 8161.074270]  [<ffffffff81169500>] ? end_buffer_async_write+0x0/0x160
[ 8161.074273]  [<ffffffff8116fd30>] ? blkdev_get_block+0x0/0x70
[ 8161.074275]  [<ffffffff81169500>] ? end_buffer_async_write+0x0/0x160
[ 8161.074278]  [<ffffffff8116bac6>] block_write_full_page_endio+0x116/0x120
[ 8161.074280]  [<ffffffff8116fd30>] ? blkdev_get_block+0x0/0x70
[ 8161.074283]  [<ffffffff8116bae5>] block_write_full_page+0x15/0x20
[ 8161.074285]  [<ffffffff8116f748>] blkdev_writepage+0x18/0x20
[ 8161.074288]  [<ffffffff810fa157>] __writepage+0x17/0x40
[ 8161.074290]  [<ffffffff810fb507>] write_cache_pages+0x267/0x3b0
[ 8161.074293]  [<ffffffff810fa140>] ? __writepage+0x0/0x40
[ 8161.074296]  [<ffffffff810fb674>] generic_writepages+0x24/0x30
[ 8161.074298]  [<ffffffff810fb6a1>] do_writepages+0x21/0x40
[ 8161.074301]  [<ffffffff811624bb>] writeback_single_inode+0xdb/0x250
[ 8161.074303]  [<ffffffff8116282d>] writeback_sb_inodes+0xdd/0x160
[ 8161.074306]  [<ffffffff81162d42>] writeback_inodes_wb+0xd2/0x110
[ 8161.074308]  [<ffffffff81162ed5>] wb_writeback+0x155/0x210
[ 8161.074311]  [<ffffffff81540044>] ? _raw_spin_lock_irqsave+0x34/0x50
[ 8161.074313]  [<ffffffff81163035>] wb_check_old_data_flush+0xa5/0xb0
[ 8161.074316]  [<ffffffff8106eb60>] ? process_timeout+0x0/0x10
[ 8161.074318]  [<ffffffff811631ea>] wb_do_writeback+0xaa/0xc0
[ 8161.074321]  [<ffffffff81163243>] bdi_writeback_task+0x43/0xc0
[ 8161.074323]  [<ffffffff8110afe0>] ? bdi_start_fn+0x0/0x100
[ 8161.074325]  [<ffffffff8110b06c>] bdi_start_fn+0x8c/0x100
[ 8161.074328]  [<ffffffff8110afe0>] ? bdi_start_fn+0x0/0x100
[ 8161.074330]  [<ffffffff8107e9d6>] kthread+0x96/0xa0
[ 8161.074333]  [<ffffffff8100be64>] kernel_thread_helper+0x4/0x10
[ 8161.074335]  [<ffffffff8107e940>] ? kthread+0x0/0xa0
[ 8161.074338]  [<ffffffff8100be60>] ? kernel_thread_helper+0x0/0x10
[ 8161.074360] INFO: task nagios3:4031 blocked for more than 120 seconds.
[ 8161.074361] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8161.074363] nagios3       D ffff880001f95780     0  4031      1 0x00000000
[ 8161.074365]  ffff880224b6dd58 0000000000000082 0000000224b6de38 ffff880224b6dfd8
[ 8161.074367]  ffff88021ff1dc00 0000000000015780 0000000000015780 ffff880224b6dfd8
[ 8161.074370]  0000000000015780 ffff880224b6dfd8 0000000000015780 ffff88021ff1dc00
[ 8161.074372] Call Trace:
[ 8161.074375]  [<ffffffff81211d1c>] start_this_handle+0x1cc/0x450
[ 8161.074378]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8161.074380]  [<ffffffff81151d30>] ? filldir+0x0/0xf0
[ 8161.074383]  [<ffffffff81151d30>] ? filldir+0x0/0xf0
[ 8161.074385]  [<ffffffff81212124>] jbd2_journal_start+0x74/0xb0
[ 8161.074389]  [<ffffffff811ec287>] ext4_journal_start_sb+0x57/0x70
[ 8161.074392]  [<ffffffff811d92ba>] ext4_dirty_inode+0x2a/0x60
[ 8161.074394]  [<ffffffff8116343a>] __mark_inode_dirty+0x3a/0x190
[ 8161.074397]  [<ffffffff811571d8>] touch_atime+0xe8/0x100
[ 8161.074400]  [<ffffffff81151bdc>] vfs_readdir+0xcc/0xd0
[ 8161.074402]  [<ffffffff81151c6a>] sys_getdents+0x8a/0xe0
[ 8161.074405]  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
[ 8172.040132] ata1: lost interrupt (Status 0x50)
[ 8172.040958] sd 0:0:0:0: [sda] Unhandled error code
[ 8172.041805] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8172.043032] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 23 73 bb 00 00 08 00
[ 8172.044503] end_request: I/O error, dev sda, sector 2323387
[ 8172.045685] Buffer I/O error on device dm-1, logical block 266270
[ 8172.047060] lost page write due to I/O error on dm-1
[ 8203.040073] ata1: lost interrupt (Status 0x50)
[ 8203.040966] sd 0:0:0:0: [sda] Unhandled error code
[ 8203.042013] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8203.043382] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 23 73 eb 00 00 08 00
[ 8203.044962] end_request: I/O error, dev sda, sector 2323435
[ 8203.046451] Buffer I/O error on device dm-1, logical block 266276
[ 8203.047575] lost page write due to I/O error on dm-1
[ 8234.040061] ata1: lost interrupt (Status 0x50)
[ 8234.041117] sd 0:0:0:0: [sda] Unhandled error code
[ 8234.042489] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8234.044019] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 23 74 13 00 00 08 00
[ 8234.045248] end_request: I/O error, dev sda, sector 2323475
[ 8234.046376] Buffer I/O error on device dm-1, logical block 266281
[ 8234.047464] lost page write due to I/O error on dm-1
[ 8265.040451] ata1: lost interrupt (Status 0x50)
[ 8265.042112] sd 0:0:0:0: [sda] Unhandled error code
[ 8265.043694] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8265.045454] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 25 27 1b 00 00 10 00
[ 8265.046746] end_request: I/O error, dev sda, sector 2434843
[ 8265.048040] Buffer I/O error on device dm-1, logical block 280202
[ 8265.049336] lost page write due to I/O error on dm-1
[ 8265.050692] Buffer I/O error on device dm-1, logical block 280203
[ 8265.052139] lost page write due to I/O error on dm-1
[ 8281.071251] INFO: task jbd2/dm-1-8:615 blocked for more than 120 seconds.
[ 8281.072269] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8281.073736] jbd2/dm-1-8   D ffff880001e95780     0   615      2 0x00000000
[ 8281.075071]  ffff880221d93d10 0000000000000046 0000000000000000 ffff880221d93fd8
[ 8281.076577]  ffff880223f11700 0000000000015780 0000000000015780 ffff880221d93fd8
[ 8281.077674]  0000000000015780 ffff880221d93fd8 0000000000015780 ffff880223f11700
[ 8281.078771] Call Trace:
[ 8281.079815]  [<ffffffff812138e7>] jbd2_journal_commit_transaction+0x1e7/0x11d0
[ 8281.080903]  [<ffffffff8100985b>] ? __switch_to+0xbb/0x2e0
[ 8281.081940]  [<ffffffff8105118e>] ? put_prev_entity+0x2e/0x80
[ 8281.083012]  [<ffffffff81540044>] ? _raw_spin_lock_irqsave+0x34/0x50
[ 8281.084074]  [<ffffffff8106eb2b>] ? lock_timer_base+0x3b/0x70
[ 8281.085120]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8281.086169]  [<ffffffff81219693>] kjournald2+0xc3/0x230
[ 8281.087213]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8281.088273]  [<ffffffff812195d0>] ? kjournald2+0x0/0x230
[ 8281.089333]  [<ffffffff8107e9d6>] kthread+0x96/0xa0
[ 8281.090401]  [<ffffffff8100be64>] kernel_thread_helper+0x4/0x10
[ 8281.091485]  [<ffffffff8107e940>] ? kthread+0x0/0xa0
[ 8281.091488]  [<ffffffff8100be60>] ? kernel_thread_helper+0x0/0x10
[ 8281.091495] INFO: task kjournald:1695 blocked for more than 120 seconds.
[ 8281.091496] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8281.091498] kjournald     D ffff880001e15780     0  1695      2 0x00000000
[ 8281.091500]  ffff880225ef7d40 0000000000000046 ffff880225ef7cf0 ffff880225ef7fd8
[ 8281.091503]  ffff8802252e5c00 0000000000015780 0000000000015780 ffff880225ef7fd8
[ 8281.091505]  0000000000015780 ffff880225ef7fd8 0000000000015780 ffff8802252e5c00
[ 8281.091507] Call Trace:
[ 8281.091510]  [<ffffffff8120bd0a>] journal_commit_transaction+0x17a/0xdf0
[ 8281.091513]  [<ffffffff8100985b>] ? __switch_to+0xbb/0x2e0
[ 8281.091515]  [<ffffffff8105118e>] ? put_prev_entity+0x2e/0x80
[ 8281.091519]  [<ffffffff81033d39>] ? default_spin_lock_flags+0x9/0x10
[ 8281.091522]  [<ffffffff81540044>] ? _raw_spin_lock_irqsave+0x34/0x50
[ 8281.091525]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8281.091527]  [<ffffffff8106f971>] ? try_to_del_timer_sync+0x51/0xe0
[ 8281.091530]  [<ffffffff81540044>] ? _raw_spin_lock_irqsave+0x34/0x50
[ 8281.091532]  [<ffffffff8120f8d3>] kjournald+0xf3/0x260
[ 8281.091535]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8281.091538]  [<ffffffff8120f7e0>] ? kjournald+0x0/0x260
[ 8281.091540]  [<ffffffff8107e9d6>] kthread+0x96/0xa0
[ 8281.091543]  [<ffffffff8100be64>] kernel_thread_helper+0x4/0x10
[ 8281.091545]  [<ffffffff8107e940>] ? kthread+0x0/0xa0
[ 8281.091547]  [<ffffffff8100be60>] ? kernel_thread_helper+0x0/0x10
[ 8281.091551] INFO: task rsyslogd:1884 blocked for more than 120 seconds.
[ 8281.091552] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8281.091553] rsyslogd      D ffff880001f95780     0  1884      1 0x00000000
[ 8281.091556]  ffff88021fe117a8 0000000000000086 ffff880215d27400 ffff88021fe11fd8
[ 8281.091558]  ffff880225c2c500 0000000000015780 0000000000015780 ffff88021fe11fd8
[ 8281.091560]  0000000000015780 ffff88021fe11fd8 0000000000015780 ffff880225c2c500
[ 8281.091562] Call Trace:
[ 8281.091565]  [<ffffffff8153e007>] io_schedule+0x47/0x70
[ 8281.091569]  [<ffffffff81169435>] sync_buffer+0x45/0x50
[ 8281.091571]  [<ffffffff8153e70a>] __wait_on_bit_lock+0x5a/0xb0
[ 8281.091573]  [<ffffffff811693f0>] ? sync_buffer+0x0/0x50
[ 8281.091576]  [<ffffffff811693f0>] ? sync_buffer+0x0/0x50
[ 8281.091578]  [<ffffffff8153e7dd>] out_of_line_wait_on_bit_lock+0x7d/0x90
[ 8281.091581]  [<ffffffff8107edf0>] ? wake_bit_function+0x0/0x40
[ 8281.091584]  [<ffffffff81168f7c>] __lock_buffer+0x2c/0x30
[ 8281.091587]  [<ffffffff8120b48a>] do_get_write_access+0x48a/0x4b0
[ 8281.091589]  [<ffffffff8120b631>] journal_get_write_access+0x31/0x50
[ 8281.091593]  [<ffffffff811c1754>] __ext3_journal_get_write_access+0x34/0x70
[ 8281.091596]  [<ffffffff811b400a>] ext3_free_data+0x3a/0x130
[ 8281.091598]  [<ffffffff811b1ea2>] ? ext3_get_branch+0xb2/0x110
[ 8281.091601]  [<ffffffff811b42c7>] ext3_free_branches+0x1c7/0x1d0
[ 8281.091604]  [<ffffffff811b535f>] ext3_truncate+0x34f/0x540
[ 8281.091606]  [<ffffffff811b6d54>] ? ext3_orphan_add+0x1c4/0x1d0
[ 8281.091609]  [<ffffffff811b57a9>] ext3_truncate_failed_write+0x29/0x30
[ 8281.091611]  [<ffffffff811b5d15>] ext3_write_begin+0x1c5/0x210
[ 8281.091614]  [<ffffffff811b4b10>] ? ext3_get_block+0x0/0x120
[ 8281.091617]  [<ffffffff810f2b51>] generic_perform_write+0xc1/0x1a0
[ 8281.091621]  [<ffffffff8116343a>] ? __mark_inode_dirty+0x3a/0x190
[ 8281.091623]  [<ffffffff810f2c95>] generic_file_buffered_write+0x65/0xa0
[ 8281.091626]  [<ffffffff810f3e01>] __generic_file_aio_write+0x251/0x3f0
[ 8281.091628]  [<ffffffff811b324d>] ? ext3_mark_iloc_dirty+0x1d/0x30
[ 8281.091631]  [<ffffffff811bc434>] ? __ext3_journal_stop+0x34/0x60
[ 8281.091634]  [<ffffffff810f3ffc>] generic_file_aio_write+0x5c/0xb0
[ 8281.091637]  [<ffffffff81141525>] do_sync_write+0xd5/0x120
[ 8281.091640]  [<ffffffff8115112a>] ? vfs_ioctl+0x3a/0xc0
[ 8281.091643]  [<ffffffff81249676>] ? security_file_permission+0x16/0x20
[ 8281.091646]  [<ffffffff811426fc>] vfs_write+0xcc/0x1a0
[ 8281.091648]  [<ffffffff811428c5>] sys_write+0x55/0x90
[ 8281.091651]  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
[ 8281.091653] INFO: task rsyslogd:1887 blocked for more than 120 seconds.
[ 8281.091654] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8281.091656] rsyslogd      D ffff880001f95780     0  1887      1 0x00000000
[ 8281.091658]  ffff88021fe39c98 0000000000000086 ffff880225ce9700 ffff88021fe39fd8
[ 8281.091660]  ffff880225ce9700 0000000000015780 0000000000015780 ffff88021fe39fd8
[ 8281.091662]  0000000000015780 ffff88021fe39fd8 0000000000015780 ffff880225ce9700
[ 8281.091664] Call Trace:
[ 8281.091667]  [<ffffffff8120a0e6>] start_this_handle+0x1b6/0x3e0
[ 8281.091670]  [<ffffffff8107edb0>] ? autoremove_wake_function+0x0/0x40
[ 8281.091672]  [<ffffffff8120a484>] journal_start+0x74/0xb0
[ 8281.091675]  [<ffffffff811bd26f>] ext3_journal_start_sb+0x4f/0x60
[ 8281.091678]  [<ffffffff811b3c1d>] ext3_dirty_inode+0x3d/0xa0
[ 8281.091680]  [<ffffffff8116343a>] __mark_inode_dirty+0x3a/0x190
[ 8281.091683]  [<ffffffff8115922c>] inode_setattr+0x7c/0x170
[ 8281.091686]  [<ffffffff811b3d92>] ext3_setattr+0x112/0x240
[ 8281.091688]  [<ffffffff81159499>] notify_change+0x179/0x310
[ 8281.091691]  [<ffffffff8113fc18>] chown_common+0x98/0xc0
[ 8281.091693]  [<ffffffff81140f18>] sys_fchown+0x98/0xc0
[ 8281.091696]  [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
[ 8296.040039] ata1: lost interrupt (Status 0x50)
[ 8296.040954] sd 0:0:0:0: [sda] Unhandled error code
[ 8296.041811] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8296.042724] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 25 27 bb 00 00 08 00
[ 8296.044216] end_request: I/O error, dev sda, sector 2435003
[ 8296.045715] Buffer I/O error on device dm-1, logical block 280222
[ 8296.047430] lost page write due to I/O error on dm-1
[ 8327.040037] ata1: lost interrupt (Status 0x50)
[ 8327.041230] sd 0:0:0:0: [sda] Unhandled error code
[ 8327.042897] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8327.044609] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 25 27 fb 00 00 08 00
[ 8327.046003] end_request: I/O error, dev sda, sector 2435067
[ 8327.047078] Buffer I/O error on device dm-1, logical block 280230
[ 8327.048147] lost page write due to I/O error on dm-1
[ 8358.040057] ata1: lost interrupt (Status 0x50)
[ 8358.041244] sd 0:0:0:0: [sda] Unhandled error code
[ 8358.042272] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8358.043339] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 25 28 23 00 00 08 00
[ 8358.044398] end_request: I/O error, dev sda, sector 2435107
[ 8358.045399] Buffer I/O error on device dm-1, logical block 280235
[ 8358.046568] lost page write due to I/O error on dm-1
[ 8389.040043] ata1: lost interrupt (Status 0x50)
[ 8389.041064] sd 0:0:0:0: [sda] Unhandled error code
[ 8389.042040] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8389.043055] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 26 da d3 00 00 08 00
[ 8389.044079] end_request: I/O error, dev sda, sector 2546387
[ 8389.045146] Buffer I/O error on device dm-1, logical block 294145
[ 8389.046705] lost page write due to I/O error on dm-1
[ 8420.040043] ata1: lost interrupt (Status 0x50)
[ 8420.041484] sd 0:0:0:0: [sda] Unhandled error code
[ 8420.043109] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8420.044625] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 26 da f3 00 00 08 00
[ 8420.045746] end_request: I/O error, dev sda, sector 2546419
[ 8420.046781] Buffer I/O error on device dm-1, logical block 294149
[ 8420.047868] lost page write due to I/O error on dm-1
[ 8451.040050] ata1: lost interrupt (Status 0x50)
[ 8451.041288] sd 0:0:0:0: [sda] Unhandled error code
[ 8451.042557] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8451.043795] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 26 db 03 00 00 08 00
[ 8451.045299] end_request: I/O error, dev sda, sector 2546435
[ 8451.046669] Buffer I/O error on device dm-1, logical block 294151
[ 8451.047880] lost page write due to I/O error on dm-1
[ 8482.040048] ata1: lost interrupt (Status 0x50)
[ 8482.041454] sd 0:0:0:0: [sda] Unhandled error code
[ 8482.043283] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8482.045019] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 26 db 23 00 00 08 00
[ 8482.046417] end_request: I/O error, dev sda, sector 2546467
[ 8482.047557] Buffer I/O error on device dm-1, logical block 294155
[ 8482.048852] lost page write due to I/O error on dm-1
[ 8513.011293] ata1: lost interrupt (Status 0x50)
[ 8513.012616] sd 0:0:0:0: [sda] Unhandled error code
[ 8513.013995] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8513.015583] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 26 db bb 00 00 08 00
[ 8513.017064] end_request: I/O error, dev sda, sector 2546619
[ 8513.018275] Buffer I/O error on device dm-1, logical block 294174
[ 8513.019508] lost page write due to I/O error on dm-1
[ 8543.041376] ata1: lost interrupt (Status 0x50)
[ 8543.042643] sd 0:0:0:0: [sda] Unhandled error code
[ 8543.043859] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8543.045648] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 26 db db 00 00 08 00
[ 8543.047630] end_request: I/O error, dev sda, sector 2546651
[ 8543.049434] Buffer I/O error on device dm-1, logical block 294178
[ 8543.050928] lost page write due to I/O error on dm-1
[ 8574.040180] ata1: lost interrupt (Status 0x50)
[ 8574.041671] sd 0:0:0:0: [sda] Unhandled error code
[ 8574.043357] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
[ 8574.045391] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 00 48 91 cb 00 00 b0 00
[ 8574.046957] end_request: I/O error, dev sda, sector 4755915
[ 8574.048199] Aborting journal on device dm-1-8.
[ 8574.049756] EXT4-fs error (device dm-1) in ext4_orphan_add: Journal has aborted
[ 8574.050325] EXT4-fs error (device dm-1): ext4_journal_start_sb: Detected aborted journal
[ 8574.050327] EXT4-fs (dm-1): Remounting filesystem read-only
[ 8574.053432] EXT4-fs error (device dm-1) in add_dirent_to_buf: Journal has aborted
[ 8574.055376] EXT4-fs error (device dm-1) in ext4_delete_inode: Journal has aborted

-- 
Lost IRQ drop harddisk
https://bugs.launchpad.net/bugs/577785
You received this bug notification because you are a member of Kernel
Bugs, which is subscribed to linux in ubuntu.




More information about the kernel-bugs mailing list