[10:43:56.758] src/tools/ci/cores_backtrace.sh linux /tmp/cores [10:43:57.462] dumping /tmp/cores/cp-3-56847.core for /usr/bin/cp [10:43:59.856] Downloading separate debug info for /usr/bin/cp... [10:43:59.856] Downloading separate debug info for /.cache/debuginfod_client/24a5a30c91a15a10306b4151db4f3c1ad1615f57/debuginfo... [10:44:00.321] [New LWP 56847] [10:44:00.322] Core was generated by `cp /tmp/cirrus-ci-build/src/test/recovery/tmp_check/t_040_standby_failover_slot'. [10:44:00.322] Program terminated with signal SIGQUIT, Quit. [10:44:00.322] #0 0x00007f3ee88b4090 in ?? () [10:44:00.324] [10:44:00.324] Thread 1 (LWP 56847): [10:44:00.324] #0 0x00007f3ee88b4090 in ?? () [10:44:00.324] No symbol table info available. [10:44:00.324] #1 0x0000000000000003 in ?? () [10:44:00.324] No symbol table info available. [10:44:00.324] #2 0x00007ffe1469e10f in ?? () [10:44:00.324] No symbol table info available. [10:44:00.324] #3 0x00007ffe1469e112 in ?? () [10:44:00.324] No symbol table info available. [10:44:00.324] #4 0x00007ffe1469e196 in ?? () [10:44:00.324] No symbol table info available. [10:44:00.324] #5 0x0000000000000000 in ?? () [10:44:00.324] No symbol table info available. [10:44:00.330] -e [10:44:00.330] [10:44:00.330] [10:44:00.872] dumping /tmp/cores/postgres-6-57044.core for /tmp/cirrus-ci-build/tmp_install/usr/local/pgsql/bin/postgres [10:44:01.203] [New LWP 57044] [10:44:02.734] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [10:44:03.793] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [10:44:05.367] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [10:44:06.133] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [10:44:07.373] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [10:44:09.022] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [10:44:10.258] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [10:44:11.181] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [10:44:12.576] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [10:44:13.360] [Thread debugging using libthread_db enabled] [10:44:13.360] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [10:44:13.360] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [10:44:14.140] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [10:44:15.520] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [10:44:16.734] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [10:44:19.415] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [10:44:19.415] Downloading separate debug info for /.cache/debuginfod_client/6bc1c815422672b74fde1b1c0803e4c9effae2a7/debuginfo... [10:44:21.058] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [10:44:22.503] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [10:44:23.923] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [10:44:24.997] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [10:44:25.765] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [10:44:27.169] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [10:44:28.264] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [10:44:28.902] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [10:44:29.685] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [10:44:30.775] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [10:44:31.551] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [10:44:32.464] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [10:44:33.379] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [10:44:34.970] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [10:44:36.793] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [10:44:38.651] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [10:44:39.270] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [10:44:39.889] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [10:44:41.431] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [10:44:42.206] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [10:44:43.463] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [10:44:44.388] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [10:44:45.458] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [10:44:46.389] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [10:44:47.625] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [10:44:48.551] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [10:44:49.319] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [10:44:49.852] Core was generated by `postgres: standby: postgres postgres [local] CALL '. [10:44:49.852] Program terminated with signal SIGABRT, Aborted. [10:44:49.852] #0 0x00007f9a31f90ce1 in raise () from /lib/x86_64-linux-gnu/libc.so.6 [10:44:50.024] [10:44:50.024] Thread 1 (Thread 0x7f9a2f929380 (LWP 57044)): [10:44:50.024] #0 0x00007f9a31f90ce1 in raise () from /lib/x86_64-linux-gnu/libc.so.6 [10:44:50.024] No symbol table info available. [10:44:50.024] #1 0x00007f9a31f7a537 in abort () from /lib/x86_64-linux-gnu/libc.so.6 [10:44:50.024] No symbol table info available. [10:44:50.024] #2 0x00005655401c5902 in ExceptionalCondition (conditionName=conditionName@entry=0x56554046aae0 "MyProc->xmin == InvalidTransactionId", fileName=fileName@entry=0x56554046a5a0 "waitlsn.c", lineNumber=lineNumber@entry=342) at assert.c:66 [10:44:50.024] No locals. [10:44:50.024] #3 0x000056553f943b34 in pg_wal_replay_wait (fcinfo=) at waitlsn.c:342 [10:44:50.024] target_lsn = 50341200 [10:44:50.024] timeout = 0 [10:44:50.024] context = [10:44:50.024] __func__ = "pg_wal_replay_wait" [10:44:50.024] #4 0x000056553f85ba94 in ExecuteCallStmt (stmt=stmt@entry=0x625000005d78, params=params@entry=0x0, atomic=, dest=dest@entry=0x625000006348) at functioncmds.c:2284 [10:44:50.024] fcinfodata = {fcinfo = {flinfo = , context = , resultinfo = , fncollation = , isnull = , nargs = , args = 0x0}, fcinfo_data = { }} [10:44:50.024] fcinfo = 0x7fff8db4ec50 [10:44:50.024] lc = [10:44:50.024] fexpr = 0x6250000068f8 [10:44:50.024] nargs = [10:44:50.024] i = [10:44:50.024] aclresult = [10:44:50.024] flinfo = {fn_addr = , fn_oid = , fn_nargs = , fn_strict = , fn_retset = , fn_stats = , fn_extra = , fn_mcxt = , fn_expr = } [10:44:50.024] callcontext = [10:44:50.024] estate = 0x625000043a00 [10:44:50.024] econtext = 0x625000043c10 [10:44:50.024] tp = [10:44:50.024] fcusage = {fs = , save_f_total_time = {ticks = }, save_total = {ticks = }, start = {ticks = }} [10:44:50.024] retval = [10:44:50.024] __func__ = "ExecuteCallStmt" [10:44:50.024] #5 0x000056553fea212e in standard_ProcessUtility (pstmt=0x625000005e28, queryString=, readOnlyTree=, context=context@entry=PROCESS_UTILITY_TOPLEVEL, params=, queryEnv=, dest=, qc=) at ../../../src/include/nodes/nodes.h:172 [10:44:50.024] parsetree = 0x625000005d78 [10:44:50.024] isTopLevel = true [10:44:50.024] isAtomicContext = false [10:44:50.024] pstate = 0x619000003380 [10:44:50.024] readonly_flags = [10:44:50.024] __func__ = "standard_ProcessUtility" [10:44:50.024] #6 0x000056553fea28ee in ProcessUtility (pstmt=pstmt@entry=0x625000005e28, queryString=, readOnlyTree=, context=context@entry=PROCESS_UTILITY_TOPLEVEL, params=, queryEnv=, dest=0x625000006348, qc=0x7fff8db4f7b0) at utility.c:523 [10:44:50.024] No locals. [10:44:50.024] #7 0x000056553fe9d55f in PortalRunUtility (portal=portal@entry=0x62500002aa00, pstmt=pstmt@entry=0x625000005e28, isTopLevel=isTopLevel@entry=true, setHoldSnapshot=setHoldSnapshot@entry=false, dest=dest@entry=0x625000006348, qc=qc@entry=0x7fff8db4f7b0) at pquery.c:1158 [10:44:50.024] No locals. [10:44:50.024] #8 0x000056553fe9da7c in PortalRunMulti (portal=portal@entry=0x62500002aa00, isTopLevel=isTopLevel@entry=true, setHoldSnapshot=setHoldSnapshot@entry=false, dest=dest@entry=0x625000006348, altdest=altdest@entry=0x625000006348, qc=qc@entry=0x7fff8db4f7b0) at pquery.c:1315 [10:44:50.024] pstmt = 0x625000005e28 [10:44:50.024] stmtlist_item__state = {l = , i = } [10:44:50.024] active_snapshot_set = false [10:44:50.024] stmtlist_item = 0x625000006b40 [10:44:50.024] #9 0x000056553fe9e720 in PortalRun (portal=portal@entry=0x62500002aa00, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true, dest=dest@entry=0x625000006348, altdest=altdest@entry=0x625000006348, qc=) at pquery.c:791 [10:44:50.024] _save_exception_stack = 0x7fff8db4f930 [10:44:50.024] _save_context_stack = 0x0 [10:44:50.024] _local_sigjmp_buf = {{__jmpbuf = {, , , , , , , }, __mask_was_saved = , __saved_mask = {__val = { }}}} [10:44:50.024] _do_rethrow = [10:44:50.024] result = [10:44:50.024] nprocessed = [10:44:50.024] saveTopTransactionResourceOwner = 0x62900001bde8 [10:44:50.024] saveTopTransactionContext = 0x625000041100 [10:44:50.024] saveActivePortal = 0x0 [10:44:50.024] saveResourceOwner = 0x62900001bde8 [10:44:50.024] savePortalContext = 0x0 [10:44:50.024] saveMemoryContext = 0x625000041100 [10:44:50.024] __func__ = "PortalRun" [10:44:50.024] #10 0x000056553fe95dbe in exec_simple_query (query_string=query_string@entry=0x625000005200 "CALL pg_wal_replay_wait('0/3002550');") at postgres.c:1274 [10:44:50.024] cmdtaglen = [10:44:50.024] snapshot_set = [10:44:50.024] per_parsetree_context = 0x0 [10:44:50.024] plantree_list = 0x625000006b28 [10:44:50.024] parsetree = 0x625000005da8 [10:44:50.024] commandTag = [10:44:50.024] qc = {commandTag = , nprocessed = } [10:44:50.024] querytree_list = [10:44:50.024] portal = 0x62500002aa00 [10:44:50.024] receiver = 0x625000006348 [10:44:50.024] format = [10:44:50.024] cmdtagname = [10:44:50.024] parsetree_item__state = {l = , i = } [10:44:50.024] dest = DestRemote [10:44:50.024] oldcontext = 0x625000041100 [10:44:50.024] parsetree_list = 0x625000005dd8 [10:44:50.024] parsetree_item = 0x625000005df0 [10:44:50.024] save_log_statement_stats = false [10:44:50.024] was_logged = true [10:44:50.024] use_implicit_block = false [10:44:50.024] msec_str = { } [10:44:50.024] __func__ = "exec_simple_query" [10:44:50.024] #11 0x000056553fe999a6 in PostgresMain (dbname=dbname@entry=0x62900001b298 "postgres", username=username@entry=0x62900001b278 "postgres") at postgres.c:4680 [10:44:50.024] query_string = 0x625000005200 "CALL pg_wal_replay_wait('0/3002550');" [10:44:50.024] firstchar = [10:44:50.024] input_message = {data = , len = , maxlen = , cursor = } [10:44:50.024] local_sigjmp_buf = {{__jmpbuf = {, , , , , , , }, __mask_was_saved = , __saved_mask = {__val = { }}}} [10:44:50.024] send_ready_for_query = [10:44:50.024] idle_in_transaction_timeout_enabled = [10:44:50.024] idle_session_timeout_enabled = [10:44:50.024] __func__ = "PostgresMain" [10:44:50.024] #12 0x000056553fe8c360 in BackendMain (startup_data=startup_data@entry=0x7fff8db4fb00 "", startup_data_len=startup_data_len@entry=4) at backend_startup.c:101 [10:44:50.024] bsdata = 0x7fff8db4fb00 [10:44:50.024] #13 0x000056553fce1645 in postmaster_child_launch (child_type=child_type@entry=B_BACKEND, startup_data=startup_data@entry=0x7fff8db4fb00 "", startup_data_len=startup_data_len@entry=4, client_sock=client_sock@entry=0x7fff8db4fb90) at launch_backend.c:265 [10:44:50.024] pid = [10:44:50.024] #14 0x000056553fce91ee in BackendStartup (client_sock=client_sock@entry=0x7fff8db4fb90) at postmaster.c:3593 [10:44:50.024] bn = 0x625000006f30 [10:44:50.024] pid = [10:44:50.024] startup_data = {canAcceptConnections = } [10:44:50.024] __func__ = "BackendStartup" [10:44:50.024] #15 0x000056553fce9902 in ServerLoop () at postmaster.c:1674 [10:44:50.024] s = {sock = , raddr = {addr = {ss_family = , __ss_padding = { }, __ss_align = }, salen = }} [10:44:50.024] i = 0 [10:44:50.024] now = [10:44:50.024] last_lockfile_recheck_time = 1711968182 [10:44:50.024] last_touch_time = 1711968182 [10:44:50.024] events = {{pos = , events = , fd = , user_data = } } [10:44:50.024] nevents = 1 [10:44:50.024] __func__ = "ServerLoop" [10:44:50.024] #16 0x000056553fcebba2 in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x6040000006d0) at postmaster.c:1372 [10:44:50.024] opt = [10:44:50.024] status = [10:44:50.024] userDoption = [10:44:50.024] listen_addr_saved = false [10:44:50.024] output_config_variable = [10:44:50.024] __func__ = "PostmasterMain" [10:44:50.024] #17 0x000056553fa8cd55 in main (argc=4, argv=0x6040000006d0) at main.c:197 [10:44:50.026] do_check_root = [10:44:50.048] -e [10:44:50.048] [10:44:50.048] [10:44:50.581] dumping /tmp/cores/sh-3-56846.core for /bin/sh [10:44:51.548] Downloading separate debug info for /bin/sh... [10:44:52.022] [New LWP 56846] [10:44:52.505] Core was generated by `sh -c cp "/tmp/cirrus-ci-build/src/test/recovery/tmp_check/t_040_standby_failov'. [10:44:52.505] Program terminated with signal SIGQUIT, Quit. [10:44:52.505] #0 0x00007f1f5d33b746 in wait4 () from /lib/x86_64-linux-gnu/libc.so.6 [10:44:52.514] [10:44:52.514] Thread 1 (LWP 56846): [10:44:52.514] #0 0x00007f1f5d33b746 in wait4 () from /lib/x86_64-linux-gnu/libc.so.6 [10:44:52.514] No symbol table info available. [10:44:52.514] #1 0x000055dfb1981a7c in waitproc (status=0x7ffe64ef0aac, block=1) at jobs.c:1181 [10:44:52.514] oldmask = {__val = {0 , 18446744067266838271, 94419245651067}} [10:44:52.514] flags = 0 [10:44:52.514] err = [10:44:52.514] oldmask = {__val = { }} [10:44:52.514] flags = [10:44:52.514] err = [10:44:52.514] #2 waitone (block=block@entry=1, job=job@entry=0x55dfb24e9c40) at jobs.c:1057 [10:44:52.514] pid = [10:44:52.514] status = 0 [10:44:52.514] jp = [10:44:52.514] thisjob = 0x0 [10:44:52.514] state = [10:44:52.514] #3 0x000055dfb1981cea in dowait (block=1, jp=0x55dfb24e9c40) at jobs.c:1139 [10:44:52.514] gotchld = [10:44:52.514] rpid = 1 [10:44:52.514] pid = [10:44:52.514] #4 0x000055dfb19832ed in waitforjob (jp=0x55dfb24e9c40) at jobs.c:1016 [10:44:52.514] st = [10:44:52.514] #5 0x000055dfb197d334 in evalcommand (cmd=0x55dfb19979a0 , flags=0) at eval.c:930 [10:44:52.514] localvar_stop = 0x0 [10:44:52.514] file_stop = 0x55dfb24e9950 [10:44:52.514] redir_stop = 0x0 [10:44:52.514] argp = [10:44:52.514] arglist = {list = 0x55dfb19979c8 , lastp = 0x55dfb24e9aa8} [10:44:52.514] varlist = {list = 0x0, lastp = 0x7ffe64ef0c70} [10:44:52.514] argv = 0x55dfb24e9ac0 [10:44:52.514] argc = 3 [10:44:52.514] osp = [10:44:52.514] sp = [10:44:52.514] cmdentry = {cmdtype = 0, u = {index = 5, cmd = 0x5, func = 0x5}} [10:44:52.514] jp = [10:44:52.514] lastarg = 0x0 [10:44:52.514] path = [10:44:52.514] spclbltin = -1 [10:44:52.514] cmd_flag = 0 [10:44:52.514] execcmd = 0 [10:44:52.514] status = [10:44:52.514] nargv = [10:44:52.514] vflags = 0 [10:44:52.514] vlocal = [10:44:52.514] #6 0x000055dfb197c27e in evaltree (n=n@entry=0x55dfb19979a0 , flags=0) at eval.c:303 [10:44:52.514] checkexit = 0 [10:44:52.514] evalfn = [10:44:52.514] smark = {stackp = 0x55dfb19977e0 , stacknxt = 0x55dfb19979c0 "cp", stacknleft = 32} [10:44:52.514] isor = [10:44:52.514] status = [10:44:52.514] evaln = [10:44:52.514] #7 0x000055dfb197cc6a in evalstring (s=0x55dfb19977e8 "cp \"/tmp/cirrus-ci-build/src/test/recovery/tmp_check/t_040_standby_failover_slots_sync_publisher_data/archives/00000001", '0' , "6\" \"pg_wal/RECOVERYXLOG\"", flags=flags@entry=0) at eval.c:185 [10:44:52.514] i = [10:44:52.514] n = 0x55dfb19979a0 [10:44:52.514] smark = {stackp = 0x55dfb19977e0 , stacknxt = 0x55dfb1997888 "cp", stacknleft = 344} [10:44:52.514] status = 0 [10:44:52.514] #8 0x000055dfb197a67f in main (argc=3, argv=0x7ffe64ef0ee8) at main.c:177 [10:44:52.514] shinit = [10:44:52.514] state = 4 [10:44:52.514] smark = {stackp = 0x55dfb19977e0 , stacknxt = 0x55dfb19977e8 "cp \"/tmp/cirrus-ci-build/src/test/recovery/tmp_check/t_040_standby_failover_slots_sync_publisher_data/archives/00000001", '0' , "6\" \"pg_wal/RECOVERYXLOG\"", stacknleft = 504} [10:44:52.514] login =