[07:44:46.583] src/tools/ci/cores_backtrace.sh linux /tmp/cores
[07:44:47.301] dumping /tmp/cores/postgres-3-29682.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres
[07:44:49.182] [New LWP 29682]
[07:44:49.182] [New LWP 29518]
[07:44:49.182] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6...
[07:44:49.944] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2...
[07:44:50.848] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1...
[07:44:52.111] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2...
[07:44:54.706] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67...
[07:44:54.706] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo...
[07:44:56.762] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67...
[07:44:57.973] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2...
[07:44:59.614] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2...
[07:45:00.731] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1...
[07:45:01.537] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0...
[07:45:02.901] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1...
[07:45:04.591] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1...
[07:45:06.073] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0...
[07:45:06.980] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1...
[07:45:08.521] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1...
[07:45:10.057] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0...
[07:45:10.535] [Thread debugging using libthread_db enabled]
[07:45:10.535] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[07:45:11.782] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2...
[07:45:13.213] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3...
[07:45:14.278] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3...
[07:45:14.917] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2...
[07:45:15.761] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0...
[07:45:18.050] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6...
[07:45:19.005] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1...
[07:45:19.835] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2...
[07:45:21.118] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2...
[07:45:22.080] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2...
[07:45:23.771] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30...
[07:45:24.888] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5...
[07:45:25.690] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1...
[07:45:27.129] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20...
[07:45:27.769] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1...
[07:45:29.297] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0...
[07:45:30.060] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0...
[07:45:31.284] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2...
[07:45:32.093] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6...
[07:45:33.212] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8...
[07:45:34.178] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6...
[07:45:35.456] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10...
[07:45:36.095] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0...
[07:45:37.066] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0...
[07:45:37.862] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7...
[07:45:38.364] Core was generated by `postgres: cascade: startup recovering 000000020000000000000004                '.
[07:45:38.364] Program terminated with signal SIGQUIT, Quit.
[07:45:38.364] #0  0x00007fe0ac1d0fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6
[07:45:38.394] [Current thread is 1 (LWP 29682)]
[07:45:38.519] 
[07:45:38.519] Thread 2 (Thread 0x7fe0a9b67a40 (LWP 29518)):
[07:45:38.519] #0  0x00007fe0ac293a61 in clone () from /lib/x86_64-linux-gnu/libc.so.6
[07:45:38.519] No symbol table info available.
[07:45:38.519] #1  0x00007fe0ac281e5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
[07:45:38.519] No symbol table info available.
[07:45:38.519] #2  0x00007fe0ac282497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
[07:45:38.519] No symbol table info available.
[07:45:38.519] #3  0x00007fe0ac281cdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6
[07:45:38.519] No symbol table info available.
[07:45:38.519] #4  0x00007fe0ac1dda69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
[07:45:38.519] No symbol table info available.
[07:45:38.519] #5  0x00005558d5c47606 in RestoreArchivedFile (path=path@entry=0x7ffc97e475d0 "", xlogfname=xlogfname@entry=0x7ffc97e47590 "00000003.history", recovername=recovername@entry=0x5558d671e650 "RECOVERYHISTORY", expectedSize=expectedSize@entry=0, cleanupEnabled=cleanupEnabled@entry=false) at ../src/backend/access/transam/xlogarchive.c:177
[07:45:38.519]         xlogpath = "pg_wal/RECOVERYHISTORY\000\000\b\000\000\000\000\000\000\064\000\000\000\000\000\000\000\000\306LX\326\000\000\000\063\020t\344\227\374\177\000\000\b\000\000\000\000\000\000\000\034\256q\326XU\000\000\b\000\000\000\000\000\000\000\231Sq\326XU\000\000\270t\344\227\374\177\000\000\225Sq\326XU\000\000\362GX\326XU\000\000\241Sq\326XU\000\000\231Sq\326XU\000\000\270t\344\227\374\177\000\000\225Sq\326XU\000\000\030y\266\251\340\177\000\000\220QX\326XU", '\000' <repeats 14 times>, "\b", '\000' <repeats 11 times>, "\060", '\000' <repeats 11 times>, "\002\000\000\000\000\000\000\000"...
[07:45:38.519]         xlogRestoreCmd = 0x5558d866d998 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/025_stuck_on_old_timeline/data/t_025_stuck_on_old_timeline_primary_data/archives/00000003.history\" \"pg_wal/RECOVERYHISTORY\""
[07:45:38.519]         lastRestartPointFname = '0' <repeats 24 times>, "\000\346q\326XU\000\000Xp\344\227\374\177\000\000\003\346q\326XU\000\000\030y\266\251\340\177\000\000NRX\326XU", '\000' <repeats 14 times>, "\b", '\000' <repeats 11 times>, "\060", '\000' <repeats 11 times>, "\002", '\000' <repeats 99 times>...
[07:45:38.519]         rc = <optimized out>
[07:45:38.519]         stat_buf = {st_dev = 3472113003596310384, st_ino = 206966894640, st_nlink = 34184295202646376, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 8, st_size = 5922071200885860208, st_blksize = 6366210006019162949, st_blocks = 93836450156364, st_atim = {tv_sec = 93840041592818, tv_nsec = 93840043234209}, st_mtim = {tv_sec = 3747088730015486873, tv_nsec = 140722856816648}, st_ctim = {tv_sec = 3597751189, tv_nsec = 140602896709912}, __glibc_reserved = {8, 93840043257372, 140722856816728}}
[07:45:38.519]         restartSegNo = <optimized out>
[07:45:38.519]         restartRedoPtr = 93840041594054
[07:45:38.519]         restartTli = 0
[07:45:38.519]         __func__ = "RestoreArchivedFile"
[07:45:38.519] #6  0x00005558d5c1da05 in existsTimeLineHistory (probeTLI=probeTLI@entry=3) at ../src/backend/access/transam/timeline.c:235
[07:45:38.519]         path = '\000' <repeats 16 times>, "\002\000\000\000\000\000\000\000\005e\305\325XU\000\000\200y\344\227\374\177\000\000\a\000\000\000\r\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000z\344\227\374\177\000\000\260y\344\227\374\177\000\000\001\000\000\000\000\000\000\000\004\000\000\t", '\000' <repeats 12 times>, "\204w\344\227\374\177\000\000\370\026}\326XU\000\000э\r\327XU\000\000@\216\r\327XU\000\000\240\037\000\000\377\377\002\000\a\027}\326XU\000\000@\216\r\327XU\000\000\002\000\000\000\000\000\000\000\357aX\326XU\000\000\246w\344\227\374\177\000\000\244w\344\227\374\177", '\000' <repeats 26 times>...
[07:45:38.519]         histfname = "00000003.history", '\000' <repeats 47 times>
[07:45:38.519]         fd = <optimized out>
[07:45:38.519]         __func__ = "existsTimeLineHistory"
[07:45:38.519] #7  0x00005558d5c1daf0 in findNewestTimeLine (startTLI=startTLI@entry=2) at ../src/backend/access/transam/timeline.c:277
[07:45:38.519]         newestTLI = 2
[07:45:38.519]         probeTLI = 3
[07:45:38.519] #8  0x00005558d5c525d1 in rescanLatestTimeLine (replayTLI=replayTLI@entry=2, replayLSN=replayLSN@entry=67211528) at ../src/backend/access/transam/xlogrecovery.c:4113
[07:45:38.519]         newExpectedTLEs = <optimized out>
[07:45:38.519]         found = <optimized out>
[07:45:38.519]         cell = <optimized out>
[07:45:38.519]         newtarget = <optimized out>
[07:45:38.519]         oldtarget = 2
[07:45:38.519]         currentTle = 0x0
[07:45:38.519]         __func__ = "rescanLatestTimeLine"
[07:45:38.519] #9  0x00005558d5c57781 in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=67211552, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=67211528, replayTLI=2, replayLSN=67211528, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3674
[07:45:38.519]         oldSource = XLOG_FROM_STREAM
[07:45:38.519]         startWalReceiver = false
[07:45:38.519]         last_fail_time = 764149312718523
[07:45:38.519]         now = <optimized out>
[07:45:38.519]         streaming_reply_sent = true
[07:45:38.519]         __func__ = "WaitForWALToBecomeAvailable"
[07:45:38.519] #10 0x00005558d5c581b7 in XLogPageRead (xlogreader=0x5558d861af40, targetPagePtr=67207168, reqLen=4384, targetRecPtr=67211528, readBuf=0x5558d866a168 "\024\321\006") at ../src/backend/access/transam/xlogrecovery.c:3351
[07:45:38.519]         private = 0x5558d861af20
[07:45:38.519]         emode = 15
[07:45:38.519]         targetPageOff = 98304
[07:45:38.519]         targetSegNo = 4
[07:45:38.519]         r = <optimized out>
[07:45:38.519]         __func__ = "XLogPageRead"
[07:45:38.519] #11 0x00005558d5c60612 in ReadPageInternal (state=state@entry=0x5558d861af40, pageptr=pageptr@entry=67207168, reqLen=reqLen@entry=4384) at ../src/backend/access/transam/xlogreader.c:1060
[07:45:38.519]         readLen = <optimized out>
[07:45:38.519]         targetPageOff = 98304
[07:45:38.519]         targetSegNo = 4
[07:45:38.519]         hdr = <optimized out>
[07:45:38.519] #12 0x00005558d5c6226d in XLogDecodeNextRecord (state=state@entry=0x5558d861af40, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:597
[07:45:38.519]         RecPtr = 67211528
[07:45:38.519]         record = <optimized out>
[07:45:38.519]         targetPagePtr = 67207168
[07:45:38.519]         randAccess = false
[07:45:38.519]         len = <optimized out>
[07:45:38.519]         total_len = <optimized out>
[07:45:38.519]         targetRecOff = 4360
[07:45:38.519]         pageHeaderSize = <optimized out>
[07:45:38.519]         assembled = false
[07:45:38.519]         gotheader = <optimized out>
[07:45:38.519]         readOff = <optimized out>
[07:45:38.519]         decoded = 0x0
[07:45:38.519]         errormsg = 0x5558d67bfc96 "LOG"
[07:45:38.519] #13 0x00005558d5c62e0f in XLogReadAhead (state=0x5558d861af40, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:973
[07:45:38.519]         result = <optimized out>
[07:45:38.519] #14 0x00005558d5c4d751 in XLogPrefetcherNextBlock (pgsr_private=93840075767152, lsn=0x5558d867fc98) at ../src/backend/access/transam/xlogprefetcher.c:492
[07:45:38.519]         nonblocking = <optimized out>
[07:45:38.519]         record = <optimized out>
[07:45:38.519]         prefetcher = 0x5558d861bd70
[07:45:38.519]         reader = 0x5558d861af40
[07:45:38.519]         replaying_lsn = 67211120
[07:45:38.519]         __func__ = "XLogPrefetcherNextBlock"
[07:45:38.519] #15 0x00005558d5c51b1a in lrq_prefetch (lrq=<optimized out>) at ../src/backend/access/transam/xlogprefetcher.c:252
[07:45:38.519] No locals.
[07:45:38.519] #16 lrq_complete_lsn (lsn=0, lrq=0x5558d867fa18) at ../src/backend/access/transam/xlogprefetcher.c:290
[07:45:38.519] No locals.
[07:45:38.519] #17 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x5558d861bd70, errmsg=errmsg@entry=0x7ffc97e4bd18) at ../src/backend/access/transam/xlogprefetcher.c:1037
[07:45:38.519]         record = <optimized out>
[07:45:38.519]         replayed_up_to = 0
[07:45:38.519] #18 0x00005558d5c589e4 in ReadRecord (xlogprefetcher=0x5558d861bd70, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=2) at ../src/backend/access/transam/xlogrecovery.c:3151
[07:45:38.519]         errormsg = 0x5558d861b750 ""
[07:45:38.519]         record = <optimized out>
[07:45:38.519]         xlogreader = 0x5558d861af40
[07:45:38.519]         private = <optimized out>
[07:45:38.519]         __func__ = "ReadRecord"
[07:45:38.519] #19 0x00005558d5c5bb7b in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1832
[07:45:38.519]         xtime = <optimized out>
[07:45:38.519]         ru0 = {tv = {tv_sec = 1710834112, tv_usec = 175653}, ru = {ru_utime = {tv_sec = 0, tv_usec = 0}, ru_stime = {tv_sec = 0, tv_usec = 2277}, {ru_maxrss = 16236, __ru_maxrss_word = 16236}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 406, __ru_minflt_word = 406}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 11, __ru_nvcsw_word = 11}, {ru_nivcsw = 2, __ru_nivcsw_word = 2}}}
[07:45:38.519]         record = 0x7fe0a943ae68
[07:45:38.539]         reachedRecoveryTarget = false
[07:45:38.539]         replayTLI = 2
[07:45:38.539]         __func__ = "PerformWalRecovery"
[07:45:38.539] #20 0x00005558d5c45828 in StartupXLOG () at ../src/backend/access/transam/xlog.c:5723
[07:45:38.539]         Insert = <optimized out>
[07:45:38.539]         checkPoint = {redo = 33554472, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 738}, nextOid = 14205, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1710834110, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 738}
[07:45:38.539]         wasShutdown = false
[07:45:38.539]         didCrash = <optimized out>
[07:45:38.539]         haveTblspcMap = false
[07:45:38.539]         haveBackupLabel = true
[07:45:38.539]         EndOfLog = <optimized out>
[07:45:38.539]         EndOfLogTLI = <optimized out>
[07:45:38.539]         newTLI = <optimized out>
[07:45:38.539]         performedWalRecovery = <optimized out>
[07:45:38.539]         endOfRecoveryInfo = <optimized out>
[07:45:38.539]         abortedRecPtr = <optimized out>
[07:45:38.539]         missingContrecPtr = <optimized out>
[07:45:38.539]         oldestActiveXID = <optimized out>
[07:45:38.539]         promoted = false
[07:45:38.539]         __func__ = "StartupXLOG"
[07:45:38.539] #21 0x00005558d60f614f in StartupProcessMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at ../src/backend/postmaster/startup.c:258
[07:45:38.539] No locals.
[07:45:38.539] #22 0x00005558d60ed919 in postmaster_child_launch (child_type=child_type@entry=B_STARTUP, startup_data=startup_data@entry=0x0, startup_data_len=startup_data_len@entry=0, client_sock=client_sock@entry=0x0) at ../src/backend/postmaster/launch_backend.c:267
[07:45:38.539]         pid = <optimized out>
[07:45:38.539] #23 0x00005558d60ef7e4 in StartChildProcess (type=type@entry=B_STARTUP) at ../src/backend/postmaster/postmaster.c:3928
[07:45:38.539]         pid = <optimized out>
[07:45:38.539]         __func__ = "StartChildProcess"
[07:45:38.539] #24 0x00005558d60f58ae in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x5558d8619e00) at ../src/backend/postmaster/postmaster.c:1364
[07:45:38.539]         opt = <optimized out>
[07:45:38.539]         status = <optimized out>
[07:45:38.539]         userDoption = <optimized out>
[07:45:38.539]         listen_addr_saved = false
[07:45:38.539]         output_config_variable = <optimized out>
[07:45:38.539]         __func__ = "PostmasterMain"
[07:45:38.539] #25 0x00005558d5f218ee in main (argc=4, argv=0x5558d8619e00) at ../src/backend/main/main.c:197
[07:45:38.539]         do_check_root = <optimized out>
[07:45:38.539] 
[07:45:38.539] Thread 1 (LWP 29682):
[07:45:38.539] #0  0x00007fe0ac1d0fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6
[07:45:38.539] No symbol table info available.
[07:45:38.539] #1  0x00007fe0ac28213a in ?? () from /lib/x86_64-linux-gnu/libc.so.6
[07:45:38.539] No symbol table info available.
[07:45:38.539] #2  0x00007fe0ac293a6f in clone () from /lib/x86_64-linux-gnu/libc.so.6
[07:45:38.539] No symbol table info available.