XFS (dm-16): Mounting Filesystem
XFS (dm-17): Mounting Filesystem
XFS (dm-16): Starting recovery (logdev: internal)
XFS (dm-16): Ending recovery (logdev: internal)
XFS (dm-17): Starting recovery (logdev: internal)
XFS (dm-17): Ending recovery (logdev: internal)
XFS (dm-4): Mounting Filesystem
XFS (dm-4): Ending clean mount
XFS (dm-5): Mounting Filesystem
XFS (dm-5): Starting recovery (logdev: internal)
XFS (dm-5): Ending recovery (logdev: internal)
XFS (dm-6): Mounting Filesystem
XFS (dm-6): Starting recovery (logdev: internal)
XFS (dm-6): Ending recovery (logdev: internal)
XFS (dm-7): Mounting Filesystem
XFS (dm-7): Starting recovery (logdev: internal)
XFS (dm-7): Ending recovery (logdev: internal)
XFS (dm-4): Mounting Filesystem
XFS (dm-4): Ending clean mount
XFS (dm-9): Mounting Filesystem
XFS (dm-9): Ending clean mount
XFS (dm-10): Mounting Filesystem
XFS (dm-10): Starting recovery (logdev: internal)
XFS (dm-10): Ending recovery (logdev: internal)
XFS (dm-11): Mounting Filesystem
XFS (dm-11): Starting recovery (logdev: internal)
XFS (dm-11): Ending recovery (logdev: internal)
XFS (dm-12): Mounting Filesystem
XFS (dm-12): Starting recovery (logdev: internal)
XFS (dm-12): Ending recovery (logdev: internal)
XFS (dm-13): Mounting Filesystem
XFS (dm-13): Starting recovery (logdev: internal)
XFS (dm-13): Ending recovery (logdev: internal)
XFS (dm-14): Mounting Filesystem
XFS (dm-14): Starting recovery (logdev: internal)
XFS (dm-14): Ending recovery (logdev: internal)
XFS (dm-15): Mounting Filesystem
XFS (dm-15): Starting recovery (logdev: internal)
XFS (dm-15): Ending recovery (logdev: internal)
XFS (dm-16): Mounting Filesystem
XFS (dm-16): Starting recovery (logdev: internal)
XFS (dm-16): Ending recovery (logdev: internal)
XFS (dm-17): Mounting Filesystem
XFS (dm-17): Starting recovery (logdev: internal)
XFS (dm-17): Ending recovery (logdev: internal)
XFS (dm-10): Mounting Filesystem
XFS (dm-10): Starting recovery (logdev: internal)
XFS (dm-10): Ending recovery (logdev: internal)
XFS (dm-11): Mounting Filesystem
XFS (dm-11): Starting recovery (logdev: internal)
XFS (dm-11): Ending recovery (logdev: internal)
XFS (dm-4): Mounting Filesystem
XFS (dm-4): Ending clean mount
XFS (dm-9): Mounting Filesystem
XFS (dm-9): Ending clean mount
XFS (dm-14): Mounting Filesystem
XFS (dm-14): Ending clean mount
XFS (dm-15): Mounting Filesystem
XFS (dm-15): Starting recovery (logdev: internal)
XFS (dm-15): Ending recovery (logdev: internal)
XFS (dm-16): Mounting Filesystem
XFS (dm-16): Starting recovery (logdev: internal)
XFS (dm-16): Ending recovery (logdev: internal)
XFS (loop1): Mounting Filesystem
XFS (loop1): Ending clean mount
XFS (loop2): Mounting Filesystem
XFS (loop2): Ending clean mount
INFO: task xfsalloc/0:463 blocked for more than 120 seconds.
      Not tainted 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
xfsalloc/0    D ffff880004b1b840     0   463      2 0x00000000
 ffff88003eb27a50 0000000000000246 0000000000000000 0000000000000000
 ffffffff810074fd ffff88003eb27a20 ffffffff81007ca2 0000000000000000
 ffff8800045705f8 ffff88003eb27fd8 000000000000fbc8 ffff8800045705f8
Call Trace:
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff8152a63c>] ? _spin_unlock_irqrestore+0x1c/0x20
 [<ffffffffa011bf62>] xlog_wait+0x72/0x90 [xfs]
 [<ffffffff81061d00>] ? default_wake_function+0x0/0x20
 [<ffffffffa011d8c3>] ? xlog_state_release_iclog+0xb3/0xf0 [xfs]
 [<ffffffffa011ddde>] _xfs_log_force+0x1ae/0x240 [xfs]
 [<ffffffffa011e038>] xfs_log_force+0x38/0x90 [xfs]
 [<ffffffffa00e3008>] xfs_alloc_ag_vextent_size+0x1b8/0x630 [xfs]
 [<ffffffffa00e404d>] xfs_alloc_ag_vextent+0xad/0x100 [xfs]
 [<ffffffffa00e4a9c>] xfs_alloc_vextent+0x2bc/0x610 [xfs]
 [<ffffffffa00ef197>] xfs_bmap_btalloc+0x267/0x700 [xfs]
 [<ffffffffa00ef63e>] xfs_bmap_alloc+0xe/0x10 [xfs]
 [<ffffffffa00f5622>] __xfs_bmapi_allocate+0x102/0x340 [xfs]
 [<ffffffff81527bfe>] ? thread_return+0x4e/0x760
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffffa00f59a0>] xfs_bmapi_allocate_worker+0x70/0xa0 [xfs]
 [<ffffffffa00f5930>] ? xfs_bmapi_allocate_worker+0x0/0xa0 [xfs]
 [<ffffffff81094980>] worker_thread+0x170/0x2a0
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff8109af00>] ? autoremove_wake_function+0x0/0x40
 [<ffffffff8152a63c>] ? _spin_unlock_irqrestore+0x1c/0x20
 [<ffffffff81094810>] ? worker_thread+0x0/0x2a0
 [<ffffffff8109ab56>] kthread+0x96/0xa0
 [<ffffffff81094810>] ? worker_thread+0x0/0x2a0
 [<ffffffff8100c20a>] child_rip+0xa/0x20
 [<ffffffff8100b294>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8100ba1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
INFO: task flush-7:0:23981 blocked for more than 120 seconds.
      Not tainted 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
flush-7:0     D ffff880004b1b840     0 23981      2 0x00000080
 ffff88003eb6f970 0000000000000246 00000000ffffffff 003eccac90e64182
 ffff88003eb6f8f0 ffff88003c28d2f0 0000000001063a6a ffffffffac679ecd
 ffff88003e018638 ffff88003eb6ffd8 000000000000fbc8 ffff88003e018638
Call Trace:
 [<ffffffff810a6d01>] ? ktime_get_ts+0xb1/0xf0
 [<ffffffff8111f590>] ? sync_page+0x0/0x50
 [<ffffffff81528383>] io_schedule+0x73/0xc0
 [<ffffffff8111f5cd>] sync_page+0x3d/0x50
 [<ffffffff81528c1a>] __wait_on_bit_lock+0x5a/0xc0
 [<ffffffff8111f567>] __lock_page+0x67/0x70
 [<ffffffff8109af80>] ? wake_bit_function+0x0/0x50
 [<ffffffff81135875>] ? pagevec_lookup_tag+0x25/0x40
 [<ffffffff811347cd>] write_cache_pages+0x3cd/0x4c0
 [<ffffffff81133320>] ? __writepage+0x0/0x40
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff811348e4>] generic_writepages+0x24/0x30
 [<ffffffffa01338ad>] xfs_vm_writepages+0x5d/0x80 [xfs]
 [<ffffffff81134911>] do_writepages+0x21/0x40
 [<ffffffff811b4c8d>] writeback_single_inode+0xdd/0x290
 [<ffffffff811b508d>] writeback_sb_inodes+0xbd/0x170
 [<ffffffff811b51eb>] writeback_inodes_wb+0xab/0x1b0
 [<ffffffff811b55e3>] wb_writeback+0x2f3/0x410
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81084a02>] ? del_timer_sync+0x22/0x30
 [<ffffffff811b58a5>] wb_do_writeback+0x1a5/0x240
 [<ffffffff811b59a3>] bdi_writeback_task+0x63/0x1b0
 [<ffffffff8109ad87>] ? bit_waitqueue+0x17/0xd0
 [<ffffffff81143590>] ? bdi_start_fn+0x0/0x100
 [<ffffffff81143616>] bdi_start_fn+0x86/0x100
 [<ffffffff81143590>] ? bdi_start_fn+0x0/0x100
 [<ffffffff8109ab56>] kthread+0x96/0xa0
 [<ffffffff8100c20a>] child_rip+0xa/0x20
 [<ffffffff8100b294>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8100ba1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
INFO: task loop1:15077 blocked for more than 120 seconds.
      Not tainted 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
loop1         D ffff880004b1b840     0 15077      2 0x00000080
 ffff88003e461500 0000000000000246 ffff88003e4614b0 ffffffffa00ed42d
 000000000000000c ffff88003d513dc0 ffffea00005b7920 00000004005b78f8
 ffff88003e454638 ffff88003e461fd8 000000000000fbc8 ffff88003e454638
Call Trace:
 [<ffffffffa00ed42d>] ? xfs_bmap_search_multi_extents+0xad/0x120 [xfs]
 [<ffffffff81528a95>] schedule_timeout+0x215/0x2e0
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff81528713>] wait_for_common+0x123/0x180
 [<ffffffff81061d00>] ? default_wake_function+0x0/0x20
 [<ffffffff81095171>] ? __queue_work+0x41/0x50
 [<ffffffff8152882d>] wait_for_completion+0x1d/0x20
 [<ffffffffa00f5908>] xfs_bmapi_allocate+0xa8/0xd0 [xfs]
 [<ffffffffa00ed42d>] ? xfs_bmap_search_multi_extents+0xad/0x120 [xfs]
 [<ffffffffa00f5930>] ? xfs_bmapi_allocate_worker+0x0/0xa0 [xfs]
 [<ffffffffa00f62b6>] xfs_bmapi+0x8e6/0xee0 [xfs]
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffffa0131fc7>] ? kmem_zone_alloc+0x77/0xf0 [xfs]
 [<ffffffffa0118b30>] xfs_iomap_write_allocate+0x140/0x370 [xfs]
 [<ffffffffa0133a83>] xfs_map_blocks+0x193/0x250 [xfs]
 [<ffffffffa0134569>] xfs_vm_writepage+0x1f9/0x590 [xfs]
 [<ffffffff81133337>] __writepage+0x17/0x40
 [<ffffffff811345fd>] write_cache_pages+0x1fd/0x4c0
 [<ffffffff81133320>] ? __writepage+0x0/0x40
 [<ffffffff811348e4>] generic_writepages+0x24/0x30
 [<ffffffffa01338ad>] xfs_vm_writepages+0x5d/0x80 [xfs]
 [<ffffffff81134911>] do_writepages+0x21/0x40
 [<ffffffff8111fd7b>] __filemap_fdatawrite_range+0x5b/0x60
 [<ffffffff810037be>] ? xen_end_context_switch+0x1e/0x30
 [<ffffffff8111fdda>] filemap_write_and_wait_range+0x5a/0x90
 [<ffffffff811ba6ce>] vfs_fsync_range+0x7e/0x100
 [<ffffffff811ba7bd>] vfs_fsync+0x1d/0x20
 [<ffffffff8137bcd7>] do_bio_filebacked+0x317/0x330
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff8152a63c>] ? _spin_unlock_irqrestore+0x1c/0x20
 [<ffffffff8137bdc1>] loop_thread+0xd1/0x270
 [<ffffffff8109af00>] ? autoremove_wake_function+0x0/0x40
 [<ffffffff8152a63c>] ? _spin_unlock_irqrestore+0x1c/0x20
 [<ffffffff8137bcf0>] ? loop_thread+0x0/0x270
 [<ffffffff8109ab56>] kthread+0x96/0xa0
 [<ffffffff8100c20a>] child_rip+0xa/0x20
 [<ffffffff8100b294>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8100ba1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
INFO: task loop2:15087 blocked for more than 120 seconds.
      Not tainted 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
loop2         D ffff880004b1b840     0 15087      2 0x00000080
 ffff88003c6a5500 0000000000000246 ffff88003c6a54b0 ffffffffa00ed42d
 000000000000000b ffff88003e6c4df0 ffffea00009af4b8 00000007009af490
 ffff880006a3b058 ffff88003c6a5fd8 000000000000fbc8 ffff880006a3b058
Call Trace:
 [<ffffffffa00ed42d>] ? xfs_bmap_search_multi_extents+0xad/0x120 [xfs]
 [<ffffffff81528a95>] schedule_timeout+0x215/0x2e0
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff81528713>] wait_for_common+0x123/0x180
 [<ffffffff81061d00>] ? default_wake_function+0x0/0x20
 [<ffffffff81095171>] ? __queue_work+0x41/0x50
 [<ffffffff8152882d>] wait_for_completion+0x1d/0x20
 [<ffffffffa00f5908>] xfs_bmapi_allocate+0xa8/0xd0 [xfs]
 [<ffffffffa00ed42d>] ? xfs_bmap_search_multi_extents+0xad/0x120 [xfs]
 [<ffffffffa00f5930>] ? xfs_bmapi_allocate_worker+0x0/0xa0 [xfs]
 [<ffffffffa00f62b6>] xfs_bmapi+0x8e6/0xee0 [xfs]
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffffa0131fc7>] ? kmem_zone_alloc+0x77/0xf0 [xfs]
 [<ffffffffa0118b30>] xfs_iomap_write_allocate+0x140/0x370 [xfs]
 [<ffffffffa0133a83>] xfs_map_blocks+0x193/0x250 [xfs]
 [<ffffffffa0134569>] xfs_vm_writepage+0x1f9/0x590 [xfs]
 [<ffffffff81133337>] __writepage+0x17/0x40
 [<ffffffff811345fd>] write_cache_pages+0x1fd/0x4c0
 [<ffffffff81133320>] ? __writepage+0x0/0x40
 [<ffffffff811348e4>] generic_writepages+0x24/0x30
 [<ffffffffa01338ad>] xfs_vm_writepages+0x5d/0x80 [xfs]
 [<ffffffff81134911>] do_writepages+0x21/0x40
 [<ffffffff8111fd7b>] __filemap_fdatawrite_range+0x5b/0x60
 [<ffffffff810037be>] ? xen_end_context_switch+0x1e/0x30
 [<ffffffff8111fdda>] filemap_write_and_wait_range+0x5a/0x90
 [<ffffffff811ba6ce>] vfs_fsync_range+0x7e/0x100
 [<ffffffff811ba7bd>] vfs_fsync+0x1d/0x20
 [<ffffffff8137bcd7>] do_bio_filebacked+0x317/0x330
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff8152a63c>] ? _spin_unlock_irqrestore+0x1c/0x20
 [<ffffffff8137bdc1>] loop_thread+0xd1/0x270
 [<ffffffff8109af00>] ? autoremove_wake_function+0x0/0x40
 [<ffffffff8152a63c>] ? _spin_unlock_irqrestore+0x1c/0x20
 [<ffffffff8137bcf0>] ? loop_thread+0x0/0x270
 [<ffffffff8109ab56>] kthread+0x96/0xa0
 [<ffffffff8100c20a>] child_rip+0xa/0x20
 [<ffffffff8100b294>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8100ba1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
INFO: task flush-7:2:15212 blocked for more than 120 seconds.
      Not tainted 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
flush-7:2     D ffff880004b1b840     0 15212      2 0x00000080
 ffff88003e1e7440 0000000000000246 0000000000000000 ffff88003eb20080
 ffff88003eb20080 ffff880004b1b840 0000000000000000 0000000000000000
 ffff88002cf17098 ffff88003e1e7fd8 000000000000fbc8 ffff88002cf17098
Call Trace:
 [<ffffffff81528a95>] schedule_timeout+0x215/0x2e0
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff81528713>] wait_for_common+0x123/0x180
 [<ffffffff81061d00>] ? default_wake_function+0x0/0x20
 [<ffffffff81095171>] ? __queue_work+0x41/0x50
 [<ffffffff8152882d>] wait_for_completion+0x1d/0x20
 [<ffffffffa00f5908>] xfs_bmapi_allocate+0xa8/0xd0 [xfs]
 [<ffffffffa00ed42d>] ? xfs_bmap_search_multi_extents+0xad/0x120 [xfs]
 [<ffffffffa00f5930>] ? xfs_bmapi_allocate_worker+0x0/0xa0 [xfs]
 [<ffffffffa00f62b6>] xfs_bmapi+0x8e6/0xee0 [xfs]
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffffa0131fc7>] ? kmem_zone_alloc+0x77/0xf0 [xfs]
 [<ffffffffa0118b30>] xfs_iomap_write_allocate+0x140/0x370 [xfs]
 [<ffffffffa0133a83>] xfs_map_blocks+0x193/0x250 [xfs]
 [<ffffffffa0134569>] xfs_vm_writepage+0x1f9/0x590 [xfs]
 [<ffffffff81133337>] __writepage+0x17/0x40
 [<ffffffff811345fd>] write_cache_pages+0x1fd/0x4c0
 [<ffffffff81135875>] ? pagevec_lookup_tag+0x25/0x40
 [<ffffffff81133320>] ? __writepage+0x0/0x40
 [<ffffffff811348e4>] generic_writepages+0x24/0x30
 [<ffffffffa01338ad>] xfs_vm_writepages+0x5d/0x80 [xfs]
 [<ffffffff81134911>] do_writepages+0x21/0x40
 [<ffffffff811b4c8d>] writeback_single_inode+0xdd/0x290
 [<ffffffff811b508d>] writeback_sb_inodes+0xbd/0x170
 [<ffffffff811b51eb>] writeback_inodes_wb+0xab/0x1b0
 [<ffffffff811b55e3>] wb_writeback+0x2f3/0x410
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81084a02>] ? del_timer_sync+0x22/0x30
 [<ffffffff811b58a5>] wb_do_writeback+0x1a5/0x240
 [<ffffffff811b59a3>] bdi_writeback_task+0x63/0x1b0
 [<ffffffff8109ad87>] ? bit_waitqueue+0x17/0xd0
 [<ffffffff81143590>] ? bdi_start_fn+0x0/0x100
 [<ffffffff81143616>] bdi_start_fn+0x86/0x100
 [<ffffffff81143590>] ? bdi_start_fn+0x0/0x100
 [<ffffffff8109ab56>] kthread+0x96/0xa0
 [<ffffffff8100c20a>] child_rip+0xa/0x20
 [<ffffffff8100b294>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8100ba1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
INFO: task flush-7:1:15213 blocked for more than 120 seconds.
      Not tainted 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
flush-7:1     D ffff880004b1b840     0 15213      2 0x00000080
 ffff88003dc014a0 0000000000000246 0000000000000000 ffff88003eb20080
 ffff88003eb20080 ffff880004b1b840 0000000000000000 0000000000000000
 ffff88002cf16638 ffff88003dc01fd8 000000000000fbc8 ffff88002cf16638
Call Trace:
 [<ffffffff81528a95>] schedule_timeout+0x215/0x2e0
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff81528713>] wait_for_common+0x123/0x180
 [<ffffffff81061d00>] ? default_wake_function+0x0/0x20
 [<ffffffff81095171>] ? __queue_work+0x41/0x50
 [<ffffffff8152882d>] wait_for_completion+0x1d/0x20
 [<ffffffffa00f5908>] xfs_bmapi_allocate+0xa8/0xd0 [xfs]
 [<ffffffffa00ed42d>] ? xfs_bmap_search_multi_extents+0xad/0x120 [xfs]
 [<ffffffffa00f5930>] ? xfs_bmapi_allocate_worker+0x0/0xa0 [xfs]
 [<ffffffffa00f62b6>] xfs_bmapi+0x8e6/0xee0 [xfs]
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffffa0131fc7>] ? kmem_zone_alloc+0x77/0xf0 [xfs]
 [<ffffffffa0118b30>] xfs_iomap_write_allocate+0x140/0x370 [xfs]
 [<ffffffffa0133a83>] xfs_map_blocks+0x193/0x250 [xfs]
 [<ffffffffa0134569>] xfs_vm_writepage+0x1f9/0x590 [xfs]
 [<ffffffff81133337>] __writepage+0x17/0x40
 [<ffffffff811345fd>] write_cache_pages+0x1fd/0x4c0
 [<ffffffff81133320>] ? __writepage+0x0/0x40
 [<ffffffff8100ba1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff811348e4>] generic_writepages+0x24/0x30
 [<ffffffffa01338ad>] xfs_vm_writepages+0x5d/0x80 [xfs]
 [<ffffffff81134911>] do_writepages+0x21/0x40
 [<ffffffff811b4c8d>] writeback_single_inode+0xdd/0x290
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff811b508d>] writeback_sb_inodes+0xbd/0x170
 [<ffffffff81007c49>] ? xen_irq_enable_direct_end+0x0/0x7
 [<ffffffff811b5464>] wb_writeback+0x174/0x410
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81084a02>] ? del_timer_sync+0x22/0x30
 [<ffffffff811b57bb>] wb_do_writeback+0xbb/0x240
 [<ffffffff811b59a3>] bdi_writeback_task+0x63/0x1b0
 [<ffffffff8109ad87>] ? bit_waitqueue+0x17/0xd0
 [<ffffffff81143590>] ? bdi_start_fn+0x0/0x100
 [<ffffffff81143616>] bdi_start_fn+0x86/0x100
 [<ffffffff81143590>] ? bdi_start_fn+0x0/0x100
 [<ffffffff8109ab56>] kthread+0x96/0xa0
 [<ffffffff8100c20a>] child_rip+0xa/0x20
 [<ffffffff8100b294>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8100ba1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
INFO: task umount:15333 blocked for more than 120 seconds.
      Not tainted 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
umount        D ffff880004b1b840     0 15333  14904 0x00000080
 ffff88003deabc28 0000000000000286 ffff88003deabbd8 ffffffff810097cc
 ffff88002cf160b8 0000000000000000 0000000000eabbe8 ffff880004b193c0
 ffff8800131abaf8 ffff88003deabfd8 000000000000fbc8 ffff8800131abaf8
Call Trace:
 [<ffffffff810097cc>] ? __switch_to+0x1ac/0x320
 [<ffffffff81527bfe>] ? thread_return+0x4e/0x760
 [<ffffffff81528a95>] schedule_timeout+0x215/0x2e0
 [<ffffffff8106a39b>] ? enqueue_task_fair+0xfb/0x100
 [<ffffffff81528713>] wait_for_common+0x123/0x180
 [<ffffffff81061d00>] ? default_wake_function+0x0/0x20
 [<ffffffff8152882d>] wait_for_completion+0x1d/0x20
 [<ffffffff811b4185>] writeback_inodes_sb_nr+0x75/0xa0
 [<ffffffff811b41f3>] writeback_inodes_sb+0x43/0x50
 [<ffffffff811ba95e>] __sync_filesystem+0x3e/0x90
 [<ffffffff811bab9a>] sync_filesystem+0x3a/0x70
 [<ffffffff8118aed7>] generic_shutdown_super+0x27/0xe0
 [<ffffffff8118afc1>] kill_block_super+0x31/0x50
 [<ffffffff8118b797>] deactivate_super+0x57/0x80
 [<ffffffff811aa73f>] mntput_no_expire+0xbf/0x110
 [<ffffffff811ab28b>] sys_umount+0x7b/0x3a0
 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task xfsalloc/0:463 blocked for more than 120 seconds.
      Not tainted 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
xfsalloc/0    D ffff880004b1b840     0   463      2 0x00000000
 ffff88003eb27a50 0000000000000246 0000000000000000 0000000000000000
 ffffffff810074fd ffff88003eb27a20 ffffffff81007ca2 0000000000000000
 ffff8800045705f8 ffff88003eb27fd8 000000000000fbc8 ffff8800045705f8
Call Trace:
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff8152a63c>] ? _spin_unlock_irqrestore+0x1c/0x20
 [<ffffffffa011bf62>] xlog_wait+0x72/0x90 [xfs]
 [<ffffffff81061d00>] ? default_wake_function+0x0/0x20
 [<ffffffffa011d8c3>] ? xlog_state_release_iclog+0xb3/0xf0 [xfs]
 [<ffffffffa011ddde>] _xfs_log_force+0x1ae/0x240 [xfs]
 [<ffffffffa011e038>] xfs_log_force+0x38/0x90 [xfs]
 [<ffffffffa00e3008>] xfs_alloc_ag_vextent_size+0x1b8/0x630 [xfs]
 [<ffffffffa00e404d>] xfs_alloc_ag_vextent+0xad/0x100 [xfs]
 [<ffffffffa00e4a9c>] xfs_alloc_vextent+0x2bc/0x610 [xfs]
 [<ffffffffa00ef197>] xfs_bmap_btalloc+0x267/0x700 [xfs]
 [<ffffffffa00ef63e>] xfs_bmap_alloc+0xe/0x10 [xfs]
 [<ffffffffa00f5622>] __xfs_bmapi_allocate+0x102/0x340 [xfs]
 [<ffffffff81527bfe>] ? thread_return+0x4e/0x760
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffffa00f59a0>] xfs_bmapi_allocate_worker+0x70/0xa0 [xfs]
 [<ffffffffa00f5930>] ? xfs_bmapi_allocate_worker+0x0/0xa0 [xfs]
 [<ffffffff81094980>] worker_thread+0x170/0x2a0
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff8109af00>] ? autoremove_wake_function+0x0/0x40
 [<ffffffff8152a63c>] ? _spin_unlock_irqrestore+0x1c/0x20
 [<ffffffff81094810>] ? worker_thread+0x0/0x2a0
 [<ffffffff8109ab56>] kthread+0x96/0xa0
 [<ffffffff81094810>] ? worker_thread+0x0/0x2a0
 [<ffffffff8100c20a>] child_rip+0xa/0x20
 [<ffffffff8100b294>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8100ba1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
INFO: task flush-7:0:23981 blocked for more than 120 seconds.
      Not tainted 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
flush-7:0     D ffff880004b1b840     0 23981      2 0x00000080
 ffff88003eb6f970 0000000000000246 00000000ffffffff 003eccac90e64182
 ffff88003eb6f8f0 ffff88003c28d2f0 0000000001063a6a ffffffffac679ecd
 ffff88003e018638 ffff88003eb6ffd8 000000000000fbc8 ffff88003e018638
Call Trace:
 [<ffffffff810a6d01>] ? ktime_get_ts+0xb1/0xf0
 [<ffffffff8111f590>] ? sync_page+0x0/0x50
 [<ffffffff81528383>] io_schedule+0x73/0xc0
 [<ffffffff8111f5cd>] sync_page+0x3d/0x50
 [<ffffffff81528c1a>] __wait_on_bit_lock+0x5a/0xc0
 [<ffffffff8111f567>] __lock_page+0x67/0x70
 [<ffffffff8109af80>] ? wake_bit_function+0x0/0x50
 [<ffffffff81135875>] ? pagevec_lookup_tag+0x25/0x40
 [<ffffffff811347cd>] write_cache_pages+0x3cd/0x4c0
 [<ffffffff81133320>] ? __writepage+0x0/0x40
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff811348e4>] generic_writepages+0x24/0x30
 [<ffffffffa01338ad>] xfs_vm_writepages+0x5d/0x80 [xfs]
 [<ffffffff81134911>] do_writepages+0x21/0x40
 [<ffffffff811b4c8d>] writeback_single_inode+0xdd/0x290
 [<ffffffff811b508d>] writeback_sb_inodes+0xbd/0x170
 [<ffffffff811b51eb>] writeback_inodes_wb+0xab/0x1b0
 [<ffffffff811b55e3>] wb_writeback+0x2f3/0x410
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81084a02>] ? del_timer_sync+0x22/0x30
 [<ffffffff811b58a5>] wb_do_writeback+0x1a5/0x240
 [<ffffffff811b59a3>] bdi_writeback_task+0x63/0x1b0
 [<ffffffff8109ad87>] ? bit_waitqueue+0x17/0xd0
 [<ffffffff81143590>] ? bdi_start_fn+0x0/0x100
 [<ffffffff81143616>] bdi_start_fn+0x86/0x100
 [<ffffffff81143590>] ? bdi_start_fn+0x0/0x100
 [<ffffffff8109ab56>] kthread+0x96/0xa0
 [<ffffffff8100c20a>] child_rip+0xa/0x20
 [<ffffffff8100b294>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8100ba1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
INFO: task loop1:15077 blocked for more than 120 seconds.
      Not tainted 2.6.32-431.17.1.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
loop1         D ffff880004b1b840     0 15077      2 0x00000080
 ffff88003e461500 0000000000000246 ffff88003e4614b0 ffffffffa00ed42d
 000000000000000c ffff88003d513dc0 ffffea00005b7920 00000004005b78f8
 ffff88003e454638 ffff88003e461fd8 000000000000fbc8 ffff88003e454638
Call Trace:
 [<ffffffffa00ed42d>] ? xfs_bmap_search_multi_extents+0xad/0x120 [xfs]
 [<ffffffff81528a95>] schedule_timeout+0x215/0x2e0
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff81528713>] wait_for_common+0x123/0x180
 [<ffffffff81061d00>] ? default_wake_function+0x0/0x20
 [<ffffffff81095171>] ? __queue_work+0x41/0x50
 [<ffffffff8152882d>] wait_for_completion+0x1d/0x20
 [<ffffffffa00f5908>] xfs_bmapi_allocate+0xa8/0xd0 [xfs]
 [<ffffffffa00ed42d>] ? xfs_bmap_search_multi_extents+0xad/0x120 [xfs]
 [<ffffffffa00f5930>] ? xfs_bmapi_allocate_worker+0x0/0xa0 [xfs]
 [<ffffffffa00f62b6>] xfs_bmapi+0x8e6/0xee0 [xfs]
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffffa0131fc7>] ? kmem_zone_alloc+0x77/0xf0 [xfs]
 [<ffffffffa0118b30>] xfs_iomap_write_allocate+0x140/0x370 [xfs]
 [<ffffffffa0133a83>] xfs_map_blocks+0x193/0x250 [xfs]
 [<ffffffffa0134569>] xfs_vm_writepage+0x1f9/0x590 [xfs]
 [<ffffffff81133337>] __writepage+0x17/0x40
 [<ffffffff811345fd>] write_cache_pages+0x1fd/0x4c0
 [<ffffffff81133320>] ? __writepage+0x0/0x40
 [<ffffffff811348e4>] generic_writepages+0x24/0x30
 [<ffffffffa01338ad>] xfs_vm_writepages+0x5d/0x80 [xfs]
 [<ffffffff81134911>] do_writepages+0x21/0x40
 [<ffffffff8111fd7b>] __filemap_fdatawrite_range+0x5b/0x60
 [<ffffffff810037be>] ? xen_end_context_switch+0x1e/0x30
 [<ffffffff8111fdda>] filemap_write_and_wait_range+0x5a/0x90
 [<ffffffff811ba6ce>] vfs_fsync_range+0x7e/0x100
 [<ffffffff811ba7bd>] vfs_fsync+0x1d/0x20
 [<ffffffff8137bcd7>] do_bio_filebacked+0x317/0x330
 [<ffffffff810074fd>] ? xen_force_evtchn_callback+0xd/0x10
 [<ffffffff81007ca2>] ? check_events+0x12/0x20
 [<ffffffff81007c8f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff8152a63c>] ? _spin_unlock_irqrestore+0x1c/0x20
 [<ffffffff8137bdc1>] loop_thread+0xd1/0x270
 [<ffffffff8109af00>] ? autoremove_wake_function+0x0/0x40
 [<ffffffff8152a63c>] ? _spin_unlock_irqrestore+0x1c/0x20
 [<ffffffff8137bcf0>] ? loop_thread+0x0/0x270
 [<ffffffff8109ab56>] kthread+0x96/0xa0
 [<ffffffff8100c20a>] child_rip+0xa/0x20
 [<ffffffff8100b294>] ? int_ret_from_sys_call+0x7/0x1b
 [<ffffffff8100ba1d>] ? retint_restore_args+0x5/0x6
 [<ffffffff8100c200>] ? child_rip+0x0/0x20
Stopping atd: [  OK  ]
Stopping sshd: [  OK  ]
Shutting down postfix: [  OK  ]
Stopping crond: [  OK  ]
Shutting down ntpd: [  OK  ]
Stopping block device availability: Deactivating block devices:
[  OK  ]
Unmounting loopback filesystems:  umount: /d: device is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1))