weixin_39581716
weixin_39581716
2021-01-07 08:23

Running `lxd sql global "describe nodes"` hangs the lxd daemon and cluster

Required information

Ubuntu 18.04


$ lxc info
config:
  core.https_address: 192.168.4.137:8443
  core.trust_password: true
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- macaroon_authentication
- network_sriov
- console
- restrict_devlxd
- migration_pre_copy
- infiniband
- maas_network
- devlxd_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
environment:
  addresses:
  - 192.168.4.137:8443
  architectures:
  - x86_64
  - i686
  certificate: |
    -----BEGIN CERTIFICATE-----
    MIIFQDCCAyigAwIBAgIQN1mUsxztsDIOXb/HYRkEWDANBgkqhkiG9w0BAQsFADAy
    MRwwGgYDVQQKExNsaW51eGNvbnRhaW5lcnMub3JnMRIwEAYDVQQDDAlyb290QG51
    YzIwHhcNMTgwNjI1MjA0NjUxWhcNMjgwNjIyMjA0NjUxWjAyMRwwGgYDVQQKExNs
    aW51eGNvbnRhaW5lcnMub3JnMRIwEAYDVQQDDAlyb290QG51YzIwggIiMA0GCSqG
    SIb3DQEBAQUAA4ICDwAwggIKAoICAQC6Hx+jUqJQMuPX5xJjMGLphFm7+LCbBXTR
    M/WWxvJCJt4IG6Eo2Bl6FF1ChntshnroggmhyNNiBKq2WyVcnaYSUplPYNlYukj8
    Pq2aw7hHeX8Jt/QV3maKad2oG9Z2rwaU+E2C380lAJwYrldwx9LErY0A8lKq2/Tg
    MemzgreC7d0I6tqLCOxPfeTrytyWw3J6CXKuwLALA6wp6SoAThsQkCUuFKrKCDxr
    ZkPqBz22OKfqZKzr1mV58o5gPMQo/BP4xQbuQotT+KyNH5IbvU96LvnPLHGXN2Ne
    OtrbxwthLm4jJQsaDSgSSgFH3sT0foMQoOW2McTnZ4bTPlR4bRLg3+Sqf43SInmq
    qBQ+M86k9osb/6gRo1GGRIqAJ54fWD7Tq04O8uV24gd5OvjtlPjs7Yc19vVnYf1C
    o812W+ybh6Z7Ze0/RA9r04HJ0ybxdCyADLHB0GQXIIlKTybOFptZ1WWmHxY/47ee
    fumBunC/L6YPlikSzJWzTD+t6GAnVCDMndOmwvH2aFk/s94zXQ7BWPJDD5idr+zE
    nn67H8quAk12Z7RCSJF9ScUyGTxzP8TmZprxGq/1BxBXbq82AlJwbu9GIDAj9STD
    lmNRNjuVKHdjKAxRhn1J9WsSLt62Di2H+x2922o+ie46iAXnRR4rpE2lH2X5qlQE
    KcSMrTjZRQIDAQABo1IwUDAOBgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAwwCgYIKwYB
    BQUHAwEwDAYDVR0TAQH/BAIwADAbBgNVHREEFDASggRudWMyhwTAqASXhwTAqATD
    MA0GCSqGSIb3DQEBCwUAA4ICAQBhS/v8VUGO6bjVVEDaPVQFAVWFUohCs6MFM/QM
    u0OcdVkOW+OpMbjmYAZIhjwF04uJuZVIrTYW9rht0f2534yJB09LzI1OoUFZn1wU
    hrL/Onygean6eATrT87EJu198BhZ7ikkTu7ZnpNFf0FBsXwnbkjwN93tD6BS34Pb
    uS+3j3ltVVIOHD7GrfBt6JJ6He7KKzewJlyOki5SZvxA/ziNPVuNQSZ4ppmB9xLd
    GRQjkdZ61KqPu74IzbaR/g1VqO8xrnYTKYlHUilJvYspGJ/BBx4uDsSS1SyRbUOP
    QRlYIsPTr+J6dBbdUMVyT9r7r/4yZwYrtZc9DKZ6gb0jJHxzx78w6rJL5qFq2UDW
    dOuIoLmJH+SReE2pWXEmE+BRgWtueFkB1NgtdXTk/qFh6OrC0OZlE1we8KFInRjY
    SiGnR6g5dfT8i5Pu1znTssmb+RS9G38ReyPJgSVB9IjbMucPHw4zCz7BLc2mq9fU
    STEuCbw9isldyW6D5/61hJxIeMFxUYH9l5uoNE/E2C3K8HPbOcFv5XRwRCiHkJsZ
    qFiGjQGCcQ7pjeBG9yMEhWmI3yyoI2iI31BAANP3BNb0lYQa8GEBacj1DU3NJGoQ
    k8m4e4MAVyOcIC4rvdgW1dDYEfaSkKAjOunNM7eh4oTDHJufdGTC1iKReSAAAzfZ
    ADqcSw==
    -----END CERTIFICATE-----
  certificate_fingerprint: c30eba58dad7a18cb4458efa04a5e3bba8e04b308d4046e680f0f16ec5707f60
  driver: lxc
  driver_version: 3.0.1
  kernel: Linux
  kernel_architecture: x86_64
  kernel_version: 4.15.0-23-generic
  server: lxd
  server_pid: 1692
  server_version: 3.0.1
  storage: btrfs
  storage_version: 4.15.1
  server_clustered: true
  server_name: nuc3

$ lxc storage list
+-------+-------------+--------+---------+---------+
| NAME  | DESCRIPTION | DRIVER |  STATE  | USED BY |
+-------+-------------+--------+---------+---------+
| local |             | btrfs  | CREATED | 2       |
+-------+-------------+--------+---------+---------+

Issue description

running lxd sql global "describe nodes" causes an lxd daemon to hang and all lxc comands to time timeout.

I'd expect to see the schema for the table.

该提问来源于开源项目:lxc/lxd

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

6条回答

  • weixin_39688875 weixin_39688875 4月前

    Nevermind, managed to reproduce it. run the query twice, the second time hangs.

    Feels like an invalid query doesn't abort the transaction or keeps the DB in a similarly locked state, blocking any further queries.

    点赞 评论 复制链接分享
  • weixin_39688875 weixin_39688875 4月前

    Pushed a fix to my branch

    点赞 评论 复制链接分享
  • weixin_39787089 weixin_39787089 4月前

    Please use:

    
    lxd sql global .schema
    

    See also https://github.com/lxc/lxd/blob/master/doc/database.md#dumping-the-database-content-or-schema.

    I'll have a look to the code, since it should return an error instead of hanging.

    点赞 评论 复制链接分享
  • weixin_39688875 weixin_39688875 4月前

    +1 on it returning an error on invalid SQL statements :)

    点赞 评论 复制链接分享
  • weixin_39787089 weixin_39787089 4月前

    I can't reproduce this issue in master:

    
    free:~$ LXD_DIR=/tmp/lxdt1/ sudo -E /home/free/go/bin/lxc cluster list
    +-------+-------------------------+----------+--------+-------------------+
    | NAME  |           URL           | DATABASE | STATE  |      MESSAGE      |
    +-------+-------------------------+----------+--------+-------------------+
    | node1 | https://10.1.1.101:8443 | YES      | ONLINE | fully operational |
    +-------+-------------------------+----------+--------+-------------------+
    | node2 | https://10.1.1.102:8443 | YES      | ONLINE | fully operational |
    +-------+-------------------------+----------+--------+-------------------+
    free:~$ LXD_DIR=/tmp/lxdt1/ sudo -E /home/free/go/bin/lxd sql global "describe nodes"
    Error: failed to exec query: near "describe": syntax error
    

    The same happens in the non-clustered case.

    点赞 评论 复制链接分享
  • weixin_39688875 weixin_39688875 4月前

    can you try again while running lxc monitor in another terminal?

    That way if the daemon hangs/crash, you may have more debug information to show us.

    点赞 评论 复制链接分享

相关推荐