[18:55:35.721] src/tools/ci/cores_backtrace.sh linux /tmp/cores [18:55:36.450] dumping /tmp/cores/postgres-3-35837.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres [18:55:38.309] [New LWP 35837] [18:55:38.309] [New LWP 34733] [18:55:38.309] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [18:55:39.082] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [18:55:40.036] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [18:55:41.297] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [18:55:43.907] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [18:55:43.907] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo... [18:55:45.998] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [18:55:47.210] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [18:55:48.891] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [18:55:50.039] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [18:55:50.874] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [18:55:52.253] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [18:55:53.996] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [18:55:55.510] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [18:55:56.433] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [18:55:57.996] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [18:55:59.597] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [18:56:00.083] [Thread debugging using libthread_db enabled] [18:56:00.083] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [18:56:01.346] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [18:56:02.819] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [18:56:03.909] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [18:56:04.567] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [18:56:05.453] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [18:56:08.022] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [18:56:09.011] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [18:56:09.852] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [18:56:11.152] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [18:56:12.154] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [18:56:13.892] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [18:56:15.041] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [18:56:15.866] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [18:56:17.363] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [18:56:18.027] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [18:56:19.599] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [18:56:20.375] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [18:56:21.623] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [18:56:22.457] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [18:56:23.617] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [18:56:24.591] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [18:56:25.879] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [18:56:26.533] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [18:56:27.502] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [18:56:28.324] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [18:56:28.829] Core was generated by `postgres: standby2: startup recovering 000000010000000000000006 '. [18:56:28.829] Program terminated with signal SIGQUIT, Quit. [18:56:28.829] #0 0x00007f1ead277fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [18:56:28.857] [Current thread is 1 (LWP 35837)] [18:56:28.973] [18:56:28.973] Thread 2 (Thread 0x7f1eaac0ea40 (LWP 34733)): [18:56:28.973] #0 0x00007f1ead33aa61 in clone () from /lib/x86_64-linux-gnu/libc.so.6 [18:56:28.973] No symbol table info available. [18:56:28.973] #1 0x00007f1ead328e5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [18:56:28.973] No symbol table info available. [18:56:28.973] #2 0x00007f1ead329497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [18:56:28.973] No symbol table info available. [18:56:28.973] #3 0x00007f1ead328cdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6 [18:56:28.973] No symbol table info available. [18:56:28.973] #4 0x00007f1ead284a69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [18:56:28.973] No symbol table info available. [18:56:28.973] #5 0x000055b1de2d47c2 in RestoreArchivedFile (path=path@entry=0x7ffe482fb570 "", xlogfname=xlogfname@entry=0x7ffe482fb530 "00000002.history", recovername=recovername@entry=0x55b1dedbbb30 "RECOVERYHISTORY", expectedSize=expectedSize@entry=0, cleanupEnabled=cleanupEnabled@entry=false) at ../src/backend/access/transam/xlogarchive.c:177 [18:56:28.973] xlogpath = "pg_wal/RECOVERYHISTORY\000\000\b\000\000\000\000\000\000\066\000\000\000\000\000\000\000\000\346\274\301\336\000\000\000\062\000\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000\374\202\333ޱU\000\000\b\000\000\000\000\000\000\000y(\333ޱU\000\000X\264/H\376\177\000\000u(\333ޱU\000\000\022\270\301ޱU\000\000\201(\333ޱU\000\000y(\333ޱU\000\000X\264/H\376\177\000\000u(\333ޱU\000\000\030\351\300\252\036\177\000\000\260\301\301ޱU", '\000' , "\b", '\000' , "\060", '\000' , "\002\000\000\000\000\000\000\000\036\000\000\000"... [18:56:28.973] xlogRestoreCmd = 0x55b1e03595e8 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/040_standby_failover_slots_sync/data/t_040_standby_failover_slots_sync_publisher_data/archives/00000002.history\" \"pg_wal/RECOVERYHISTORY\"" [18:56:28.973] lastRestartPointFname = '0' , "\000\272\333ޱU\000\000\370\257/H\376\177\000\000\343\272\333ޱU\000\000\030\351\300\252\036\177\000\000n\302\301ޱU", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' ... [18:56:28.973] rc = [18:56:28.973] stat_buf = {st_dev = 139769690515736, st_ino = 209895670382, st_nlink = 0, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 48, st_size = 5922071200885860208, st_blksize = 6366210006019162949, st_blocks = 4673356, st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 3891110078048108544, tv_nsec = 0}, st_ctim = {tv_sec = 0, tv_nsec = 0}, __glibc_reserved = {8, 94222436500220, 140730109505528}} [18:56:28.973] restartSegNo = [18:56:28.973] restartRedoPtr = 94222434811110 [18:56:28.973] restartTli = 0 [18:56:28.973] __func__ = "RestoreArchivedFile" [18:56:28.973] #6 0x000055b1de2aabd0 in existsTimeLineHistory (probeTLI=probeTLI@entry=2) at ../src/backend/access/transam/timeline.c:235 [18:56:28.973] path = '\000' , "\001\000\000\000\000\000\000\000\301\066.ޱU\000\000\240\271/H\376\177\000\000\a\000\000\000\r\000\000\000\001\000\000\000\000\000\000\000\004\000\000\t", '\000' , "R\312\301ޱU\000\000\000\000\000\000\000\000\000\000\342]y߱U\000\000`^y߱U\000\000\240\037\000\000\377\377\002\000X\374\346ޱU\000\000\361]y߱U\000\000`^y߱U\000\000D\267/H\376\177\000\000g\374\346ޱU\000\000`^y߱U\000\000\002\000\000\000\000\000\000\000\017\322\301ޱU\000\000F\267/H\376\177\000\000D\267/H\376\177", '\000' ... [18:56:28.973] histfname = "00000002.history", '\000' [18:56:28.973] fd = [18:56:28.973] __func__ = "existsTimeLineHistory" [18:56:28.973] #7 0x000055b1de2aacbb in findNewestTimeLine (startTLI=startTLI@entry=1) at ../src/backend/access/transam/timeline.c:277 [18:56:28.973] newestTLI = 1 [18:56:28.973] probeTLI = 2 [18:56:28.973] #8 0x000055b1de2df78d in rescanLatestTimeLine (replayTLI=replayTLI@entry=1, replayLSN=replayLSN@entry=100683536) at ../src/backend/access/transam/xlogrecovery.c:4113 [18:56:28.973] newExpectedTLEs = [18:56:28.973] found = [18:56:28.973] cell = [18:56:28.973] newtarget = [18:56:28.973] oldtarget = 1 [18:56:28.973] currentTle = 0x0 [18:56:28.973] __func__ = "rescanLatestTimeLine" [18:56:28.973] #9 0x000055b1de2e493d in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=100683560, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=100683536, replayTLI=1, replayLSN=100683536, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3674 [18:56:28.973] oldSource = XLOG_FROM_STREAM [18:56:28.973] startWalReceiver = false [18:56:28.973] last_fail_time = 764448793639524 [18:56:28.973] now = [18:56:28.973] streaming_reply_sent = true [18:56:28.973] __func__ = "WaitForWALToBecomeAvailable" [18:56:28.973] #10 0x000055b1de2e5373 in XLogPageRead (xlogreader=0x55b1e0306f30, targetPagePtr=100679680, reqLen=3880, targetRecPtr=100683536, readBuf=0x55b1e0355db8 "\024\321\006") at ../src/backend/access/transam/xlogrecovery.c:3351 [18:56:28.973] private = 0x55b1e0306f10 [18:56:28.973] emode = 15 [18:56:28.973] targetPageOff = 16384 [18:56:28.973] targetSegNo = 6 [18:56:28.973] r = [18:56:28.973] __func__ = "XLogPageRead" [18:56:28.973] #11 0x000055b1de2ed7ce in ReadPageInternal (state=state@entry=0x55b1e0306f30, pageptr=pageptr@entry=100679680, reqLen=reqLen@entry=3880) at ../src/backend/access/transam/xlogreader.c:1060 [18:56:28.973] readLen = [18:56:28.973] targetPageOff = 16384 [18:56:28.973] targetSegNo = 6 [18:56:28.973] hdr = [18:56:28.973] #12 0x000055b1de2ef429 in XLogDecodeNextRecord (state=state@entry=0x55b1e0306f30, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:597 [18:56:28.973] RecPtr = 100683536 [18:56:28.973] record = [18:56:28.973] targetPagePtr = 100679680 [18:56:28.973] randAccess = false [18:56:28.973] len = [18:56:28.973] total_len = [18:56:28.973] targetRecOff = 3856 [18:56:28.973] pageHeaderSize = [18:56:28.973] assembled = false [18:56:28.973] gotheader = [18:56:28.973] readOff = [18:56:28.973] decoded = 0x0 [18:56:28.973] errormsg = 0x55b1dee5e092 "LOG" [18:56:28.973] #13 0x000055b1de2effcb in XLogReadAhead (state=0x55b1e0306f30, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:973 [18:56:28.973] result = [18:56:28.973] #14 0x000055b1de2da90d in XLogPrefetcherNextBlock (pgsr_private=94222458846560, lsn=0x55b1e036b738) at ../src/backend/access/transam/xlogprefetcher.c:492 [18:56:28.973] nonblocking = [18:56:28.973] record = [18:56:28.973] prefetcher = 0x55b1e0307d60 [18:56:28.973] reader = 0x55b1e0306f30 [18:56:28.973] replaying_lsn = 100683488 [18:56:28.973] __func__ = "XLogPrefetcherNextBlock" [18:56:28.973] #15 0x000055b1de2decd6 in lrq_prefetch (lrq=) at ../src/backend/access/transam/xlogprefetcher.c:252 [18:56:28.973] No locals. [18:56:28.973] #16 lrq_complete_lsn (lsn=0, lrq=0x55b1e036b668) at ../src/backend/access/transam/xlogprefetcher.c:290 [18:56:28.973] No locals. [18:56:28.973] #17 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x55b1e0307d60, errmsg=errmsg@entry=0x7ffe482ffcb8) at ../src/backend/access/transam/xlogprefetcher.c:1037 [18:56:28.973] record = [18:56:28.973] replayed_up_to = 0 [18:56:28.973] #18 0x000055b1de2e5ba0 in ReadRecord (xlogprefetcher=0x55b1e0307d60, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=1) at ../src/backend/access/transam/xlogrecovery.c:3151 [18:56:28.973] errormsg = 0x55b1e0307740 "" [18:56:28.973] record = [18:56:28.973] xlogreader = 0x55b1e0306f30 [18:56:28.973] private = [18:56:28.973] __func__ = "ReadRecord" [18:56:28.973] #19 0x000055b1de2e8d37 in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1832 [18:56:28.973] xtime = [18:56:28.973] ru0 = {tv = {tv_sec = 1711133587, tv_usec = 757172}, ru = {ru_utime = {tv_sec = 0, tv_usec = 1341}, ru_stime = {tv_sec = 0, tv_usec = 0}, {ru_maxrss = 15680, __ru_maxrss_word = 15680}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 398, __ru_minflt_word = 398}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 4, __ru_nvcsw_word = 4}, {ru_nivcsw = 1, __ru_nivcsw_word = 1}}} [18:56:28.973] record = 0x7f1eaa4c8f70 [18:56:28.973] reachedRecoveryTarget = false [18:56:28.973] replayTLI = 1 [18:56:28.973] __func__ = "PerformWalRecovery" [18:56:28.973] #20 0x000055b1de2d29e4 in StartupXLOG () at ../src/backend/access/transam/xlog.c:5723 [18:56:28.992] Insert = [18:56:28.992] checkPoint = {redo = 83886120, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 749}, nextOid = 24576, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1711133587, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 749} [18:56:28.992] wasShutdown = false [18:56:28.992] didCrash = [18:56:28.992] haveTblspcMap = false [18:56:28.992] haveBackupLabel = true [18:56:28.992] EndOfLog = [18:56:28.992] EndOfLogTLI = [18:56:28.992] newTLI = [18:56:28.992] performedWalRecovery = [18:56:28.992] endOfRecoveryInfo = [18:56:28.992] abortedRecPtr = [18:56:28.992] missingContrecPtr = [18:56:28.992] oldestActiveXID = [18:56:28.992] promoted = false [18:56:28.992] __func__ = "StartupXLOG" [18:56:28.992] #21 0x000055b1de787b32 in StartupProcessMain (startup_data=, startup_data_len=) at ../src/backend/postmaster/startup.c:258 [18:56:28.992] No locals. [18:56:28.992] #22 0x000055b1de77f2fc in postmaster_child_launch (child_type=child_type@entry=B_STARTUP, startup_data=startup_data@entry=0x0, startup_data_len=startup_data_len@entry=0, client_sock=client_sock@entry=0x0) at ../src/backend/postmaster/launch_backend.c:265 [18:56:28.992] pid = [18:56:28.992] #23 0x000055b1de7811c7 in StartChildProcess (type=type@entry=B_STARTUP) at ../src/backend/postmaster/postmaster.c:3928 [18:56:28.992] pid = [18:56:28.992] __func__ = "StartChildProcess" [18:56:28.992] #24 0x000055b1de787291 in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x55b1e0305df0) at ../src/backend/postmaster/postmaster.c:1364 [18:56:28.992] opt = [18:56:28.992] status = [18:56:28.992] userDoption = [18:56:28.992] listen_addr_saved = false [18:56:28.992] output_config_variable = [18:56:28.992] __func__ = "PostmasterMain" [18:56:28.992] #25 0x000055b1de5b178c in main (argc=4, argv=0x55b1e0305df0) at ../src/backend/main/main.c:197 [18:56:28.992] do_check_root = [18:56:28.992] [18:56:28.992] Thread 1 (LWP 35837): [18:56:28.992] #0 0x00007f1ead277fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [18:56:28.992] No symbol table info available. [18:56:28.992] #1 0x00007f1ead32913a in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [18:56:28.992] No symbol table info available. [18:56:28.992] #2 0x00007f1ead33aa6f in clone () from /lib/x86_64-linux-gnu/libc.so.6 [18:56:28.992] No symbol table info available.