[05:14:40.460] src/tools/ci/cores_backtrace.sh linux /tmp/cores [05:14:41.173] dumping /tmp/cores/postgres-3-32828.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres [05:14:42.895] [New LWP 32828] [05:14:42.895] [New LWP 32104] [05:14:42.895] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [05:14:43.693] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [05:14:44.489] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [05:14:45.763] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [05:14:48.222] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [05:14:48.222] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo... [05:14:50.175] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [05:14:51.437] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [05:14:53.076] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [05:14:54.180] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [05:14:54.978] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [05:14:56.254] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [05:14:57.936] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [05:14:59.402] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [05:15:00.349] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [05:15:01.948] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [05:15:03.383] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [05:15:03.870] [Thread debugging using libthread_db enabled] [05:15:03.870] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [05:15:04.970] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [05:15:06.418] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [05:15:07.522] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [05:15:08.163] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [05:15:08.959] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [05:15:11.320] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [05:15:12.283] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [05:15:13.079] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [05:15:14.027] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [05:15:14.981] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [05:15:16.634] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [05:15:17.741] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [05:15:18.539] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [05:15:19.975] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [05:15:20.617] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [05:15:22.217] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [05:15:23.010] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [05:15:24.285] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [05:15:25.083] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [05:15:26.195] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [05:15:27.151] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [05:15:28.419] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [05:15:29.060] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [05:15:30.014] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [05:15:30.808] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [05:15:31.319] Core was generated by `postgres: standby1: startup recovering 000000010000000000000004 '. [05:15:31.319] Program terminated with signal SIGQUIT, Quit. [05:15:31.319] #0 0x00007fcde5a7dfc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [05:15:31.347] [Current thread is 1 (LWP 32828)] [05:15:31.453] [05:15:31.453] Thread 2 (Thread 0x7fcde3414a40 (LWP 32104)): [05:15:31.453] #0 0x00007fcde5b40a61 in clone () from /lib/x86_64-linux-gnu/libc.so.6 [05:15:31.453] No symbol table info available. [05:15:31.453] #1 0x00007fcde5b2ee5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [05:15:31.453] No symbol table info available. [05:15:31.453] #2 0x00007fcde5b2f497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [05:15:31.453] No symbol table info available. [05:15:31.453] #3 0x00007fcde5b2ecdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6 [05:15:31.453] No symbol table info available. [05:15:31.453] #4 0x00007fcde5a8aa69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [05:15:31.453] No symbol table info available. [05:15:31.453] #5 0x0000564af2b99950 in RestoreArchivedFile (path=path@entry=0x7ffe8cdcf760 "", xlogfname=xlogfname@entry=0x7ffe8cdcf720 "00000002.history", recovername=recovername@entry=0x564af36710f0 "RECOVERYHISTORY", expectedSize=expectedSize@entry=0, cleanupEnabled=cleanupEnabled@entry=false) at ../src/backend/access/transam/xlogarchive.c:178 [05:15:31.453] xlogpath = "pg_wal/RECOVERYHISTORY\000\000\247\243\275\345\315\000\000\064\000\000\000\000\000\000\000\000\000x!\236\000\000\000\062\000\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000\304\330f\363JV\000\000\b\000\000\000\000\000\000\000I\200f\363JV\000\000H\366܌\376\177\000\000E\200f\363JV\000\000\236qM\363JV\000\000Q\200f\363JV\000\000I\200f\363JV\000\000H\366܌\376\177\000\000E\200f\363JV\000\000\030IA\343\315\177\000\000<{M\363JV", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' ... [05:15:31.453] xlogRestoreCmd = 0x564af5cae8b8 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/040_standby_failover_slots_sync/data/t_040_standby_failover_slots_sync_publisher_data/archives/00000002.history\" \"pg_wal/RECOVERYHISTORY\"" [05:15:31.453] lastRestartPointFname = '0' , "\000\020g\363JV\000\000\350\361܌\376\177\000\000\243\020g\363JV\000\000\030IA\343\315\177\000\000\372{M\363JV", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' , "/tmp/cirr"... [05:15:31.453] rc = [05:15:31.453] stat_buf = {st_dev = 140522257729816, st_ino = 210240371706, st_nlink = 0, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 48, st_size = 5922071200885860208, st_blksize = 6366210006019162949, st_blocks = 4673356, st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 3746994889972252672, tv_nsec = 0}, st_ctim = {tv_sec = 0, tv_nsec = 0}, __glibc_reserved = {8, 94879911172292, 140731261710824}} [05:15:31.453] restartSegNo = [05:15:31.453] restartRedoPtr = 94879909508722 [05:15:31.453] restartTli = 0 [05:15:31.453] __func__ = "RestoreArchivedFile" [05:15:31.453] #6 0x0000564af2b6fd03 in existsTimeLineHistory (probeTLI=probeTLI@entry=2) at ../src/backend/access/transam/timeline.c:235 [05:15:31.453] path = '\000' , "\001\000\000\000\000\000\000\000O\210\272\362JV\000\000\001\000\000\000\000\000\000\000\006\000\000\000\r\000\000\000\220\373܌\376\177\000\000@\373܌\376\177\000\000\001\000\000\000\000\000\000\000\004\000\000\t", '\000' , "\202\v\002\364JV\000\000\000\f\002\364JV\000\000\024\371܌\376\177\000\000\330\070r\363JV\000\000\240\037\000\000\377\377\002\000\000\f\002\364JV\000\000\064\371܌\376\177\000\000\347\070r\363JV\000\000\000\f\002\364JV\000\000\002\000\000\000\000\000\000\000\233\213M\363JV\000\000\066\371܌\376\177\000\000\064\371܌\376\177", '\000' ... [05:15:31.453] histfname = "00000002.history", '\000' [05:15:31.453] fd = [05:15:31.453] __func__ = "existsTimeLineHistory" [05:15:31.453] #7 0x0000564af2b6fdee in findNewestTimeLine (startTLI=startTLI@entry=1) at ../src/backend/access/transam/timeline.c:277 [05:15:31.453] newestTLI = 1 [05:15:31.453] probeTLI = 2 [05:15:31.453] #8 0x0000564af2ba491b in rescanLatestTimeLine (replayTLI=replayTLI@entry=1, replayLSN=replayLSN@entry=67109168) at ../src/backend/access/transam/xlogrecovery.c:4095 [05:15:31.453] newExpectedTLEs = [05:15:31.453] found = [05:15:31.453] cell = [05:15:31.453] newtarget = [05:15:31.453] oldtarget = 1 [05:15:31.453] currentTle = 0x0 [05:15:31.453] __func__ = "rescanLatestTimeLine" [05:15:31.453] #9 0x0000564af2ba9acb in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=67109192, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=67109168, replayTLI=1, replayLSN=67109168, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3656 [05:15:31.453] oldSource = XLOG_FROM_STREAM [05:15:31.453] startWalReceiver = false [05:15:31.453] last_fail_time = 761893936220446 [05:15:31.453] now = [05:15:31.453] streaming_reply_sent = true [05:15:31.453] __func__ = "WaitForWALToBecomeAvailable" [05:15:31.453] #10 0x0000564af2baa501 in XLogPageRead (xlogreader=0x564af5c5bf10, targetPagePtr=67108864, reqLen=328, targetRecPtr=67109168, readBuf=0x564af5cab088 "\024\321\006") at ../src/backend/access/transam/xlogrecovery.c:3333 [05:15:31.453] private = 0x564af5c5bef0 [05:15:31.453] emode = 15 [05:15:31.453] targetPageOff = 0 [05:15:31.453] targetSegNo = 4 [05:15:31.453] r = [05:15:31.453] __func__ = "XLogPageRead" [05:15:31.453] #11 0x0000564af2bb2921 in ReadPageInternal (state=state@entry=0x564af5c5bf10, pageptr=pageptr@entry=67108864, reqLen=reqLen@entry=328) at ../src/backend/access/transam/xlogreader.c:1062 [05:15:31.453] readLen = [05:15:31.453] targetPageOff = 0 [05:15:31.453] targetSegNo = 4 [05:15:31.453] hdr = [05:15:31.453] #12 0x0000564af2bb457c in XLogDecodeNextRecord (state=state@entry=0x564af5c5bf10, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:599 [05:15:31.453] RecPtr = 67109168 [05:15:31.453] record = [05:15:31.453] targetPagePtr = 67108864 [05:15:31.453] randAccess = false [05:15:31.453] len = [05:15:31.453] total_len = [05:15:31.453] targetRecOff = 304 [05:15:31.453] pageHeaderSize = [05:15:31.453] assembled = false [05:15:31.453] gotheader = [05:15:31.453] readOff = [05:15:31.453] decoded = 0x0 [05:15:31.453] errormsg = 0x564af3711e67 "LOG" [05:15:31.453] #13 0x0000564af2bb511e in XLogReadAhead (state=0x564af5c5bf10, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:975 [05:15:31.453] result = [05:15:31.453] #14 0x0000564af2b9fa9b in XLogPrefetcherNextBlock (pgsr_private=94879950949696, lsn=0x564af5cc0968) at ../src/backend/access/transam/xlogprefetcher.c:496 [05:15:31.453] nonblocking = [05:15:31.453] record = [05:15:31.453] prefetcher = 0x564af5c5cd40 [05:15:31.453] reader = 0x564af5c5bf10 [05:15:31.453] replaying_lsn = 67109112 [05:15:31.453] __func__ = "XLogPrefetcherNextBlock" [05:15:31.453] #15 0x0000564af2ba3e64 in lrq_prefetch (lrq=) at ../src/backend/access/transam/xlogprefetcher.c:256 [05:15:31.453] No locals. [05:15:31.453] #16 lrq_complete_lsn (lsn=0, lrq=0x564af5cc0938) at ../src/backend/access/transam/xlogprefetcher.c:294 [05:15:31.453] No locals. [05:15:31.453] #17 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x564af5c5cd40, errmsg=errmsg@entry=0x7ffe8cdd3ea8) at ../src/backend/access/transam/xlogprefetcher.c:1041 [05:15:31.453] record = [05:15:31.453] replayed_up_to = 0 [05:15:31.453] #18 0x0000564af2baad2e in ReadRecord (xlogprefetcher=0x564af5c5cd40, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=1) at ../src/backend/access/transam/xlogrecovery.c:3133 [05:15:31.453] errormsg = 0x564af5c5c720 "" [05:15:31.453] record = [05:15:31.453] xlogreader = 0x564af5c5bf10 [05:15:31.453] private = [05:15:31.453] __func__ = "ReadRecord" [05:15:31.453] #19 0x0000564af2bade8a in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1814 [05:15:31.453] xtime = [05:15:31.453] ru0 = {tv = {tv_sec = 1708578732, tv_usec = 464312}, ru = {ru_utime = {tv_sec = 0, tv_usec = 1435}, ru_stime = {tv_sec = 0, tv_usec = 0}, {ru_maxrss = 16332, __ru_maxrss_word = 16332}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 409, __ru_minflt_word = 409}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 4, __ru_nvcsw_word = 4}, {ru_nivcsw = 1, __ru_nivcsw_word = 1}}} [05:15:31.453] record = 0x7fcde2d58070 [05:15:31.453] reachedRecoveryTarget = false [05:15:31.453] replayTLI = 1 [05:15:31.453] __func__ = "PerformWalRecovery" [05:15:31.471] #20 0x0000564af2b97bd9 in StartupXLOG () at ../src/backend/access/transam/xlog.c:5733 [05:15:31.471] Insert = [05:15:31.471] checkPoint = {redo = 33554472, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 739}, nextOid = 24576, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1708578731, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 739} [05:15:31.471] wasShutdown = false [05:15:31.471] didCrash = [05:15:31.471] haveTblspcMap = false [05:15:31.471] haveBackupLabel = true [05:15:31.471] EndOfLog = [05:15:31.471] EndOfLogTLI = [05:15:31.471] newTLI = [05:15:31.471] performedWalRecovery = [05:15:31.471] endOfRecoveryInfo = [05:15:31.471] abortedRecPtr = [05:15:31.471] missingContrecPtr = [05:15:31.471] oldestActiveXID = [05:15:31.471] promoted = false [05:15:31.471] __func__ = "StartupXLOG" [05:15:31.471] #21 0x0000564af3048352 in StartupProcessMain () at ../src/backend/postmaster/startup.c:254 [05:15:31.471] No locals. [05:15:31.471] #22 0x0000564af303883d in AuxiliaryProcessMain (auxtype=auxtype@entry=StartupProcess) at ../src/backend/postmaster/auxprocess.c:142 [05:15:31.471] __func__ = "AuxiliaryProcessMain" [05:15:31.471] #23 0x0000564af304369e in StartChildProcess (type=type@entry=StartupProcess) at ../src/backend/postmaster/postmaster.c:5311 [05:15:31.471] pid = [05:15:31.471] __func__ = "StartChildProcess" [05:15:31.471] #24 0x0000564af3047a92 in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x564af5c5ad80) at ../src/backend/postmaster/postmaster.c:1459 [05:15:31.471] opt = [05:15:31.471] status = [05:15:31.471] userDoption = [05:15:31.471] listen_addr_saved = false [05:15:31.471] output_config_variable = [05:15:31.471] __func__ = "PostmasterMain" [05:15:31.471] #25 0x0000564af2e728f4 in main (argc=4, argv=0x564af5c5ad80) at ../src/backend/main/main.c:198 [05:15:31.471] do_check_root = [05:15:31.471] [05:15:31.471] Thread 1 (LWP 32828): [05:15:31.471] #0 0x00007fcde5a7dfc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [05:15:31.471] No symbol table info available. [05:15:31.471] #1 0x00007fcde5b2f13a in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [05:15:31.471] No symbol table info available. [05:15:31.471] #2 0x00007fcde5b40a6f in clone () from /lib/x86_64-linux-gnu/libc.so.6 [05:15:31.471] No symbol table info available.