[21:50:48.580] src/tools/ci/cores_backtrace.sh linux /tmp/cores
[21:50:49.282] dumping /tmp/cores/postgres-3-20076.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres
[21:50:50.973] [New LWP 20076]
[21:50:50.973] [New LWP 19893]
[21:50:50.973] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6...
[21:50:51.750] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2...
[21:50:52.534] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1...
[21:50:53.791] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2...
[21:50:56.176] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67...
[21:50:56.176] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo...
[21:50:58.069] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67...
[21:50:59.286] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2...
[21:51:00.891] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2...
[21:51:01.972] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1...
[21:51:02.744] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0...
[21:51:03.975] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1...
[21:51:05.598] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1...
[21:51:07.030] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0...
[21:51:07.957] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1...
[21:51:09.508] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1...
[21:51:10.922] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0...
[21:51:11.402] [Thread debugging using libthread_db enabled]
[21:51:11.402] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[21:51:12.498] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2...
[21:51:13.920] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3...
[21:51:14.984] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3...
[21:51:15.598] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2...
[21:51:16.363] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0...
[21:51:18.642] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6...
[21:51:19.578] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1...
[21:51:20.343] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2...
[21:51:21.267] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2...
[21:51:22.194] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2...
[21:51:23.815] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30...
[21:51:24.905] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5...
[21:51:25.682] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1...
[21:51:27.067] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20...
[21:51:27.686] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1...
[21:51:29.233] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0...
[21:51:30.004] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0...
[21:51:31.249] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2...
[21:51:32.179] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6...
[21:51:33.261] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8...
[21:51:34.277] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6...
[21:51:35.532] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10...
[21:51:36.167] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0...
[21:51:37.096] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0...
[21:51:37.872] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7...
[21:51:38.378] Core was generated by `postgres: cascade: startup recovering 000000020000000000000004                '.
[21:51:38.378] Program terminated with signal SIGQUIT, Quit.
[21:51:38.378] #0  0x00007f2ec4a9ffc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6
[21:51:38.406] [Current thread is 1 (LWP 20076)]
[21:51:38.541] 
[21:51:38.541] Thread 2 (Thread 0x7f2ec2436a40 (LWP 19893)):
[21:51:38.541] #0  0x00007f2ec4b62a61 in clone () from /lib/x86_64-linux-gnu/libc.so.6
[21:51:38.541] No symbol table info available.
[21:51:38.541] #1  0x00007f2ec4b50e5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
[21:51:38.541] No symbol table info available.
[21:51:38.541] #2  0x00007f2ec4b51497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
[21:51:38.541] No symbol table info available.
[21:51:38.541] #3  0x00007f2ec4b50cdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6
[21:51:38.541] No symbol table info available.
[21:51:38.541] #4  0x00007f2ec4aaca69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
[21:51:38.541] No symbol table info available.
[21:51:38.541] #5  0x0000559446255ba3 in RestoreArchivedFile (path=path@entry=0x7ffc1275a490 "", xlogfname=xlogfname@entry=0x7ffc1275a450 "00000003.history", recovername=recovername@entry=0x559446d64a70 "RECOVERYHISTORY", expectedSize=expectedSize@entry=0, cleanupEnabled=cleanupEnabled@entry=false) at ../src/backend/access/transam/xlogarchive.c:177
[21:51:38.541]         xlogpath = "pg_wal/RECOVERYHISTORY\000\000\b\000\000\000\000\000\000\064\000\000\000\000\000\000\000\000,\253\273F\000\000\000\063Тu\022\374\177\000\000\b\000\000\000\000\000\000\000L\022\326F\224U\000\000\b\000\000\000\000\000\000\000\071\270\325F\224U\000\000x\243u\022\374\177\000\000\065\270\325F\224U\000\000X\246\273F\224U\000\000A\270\325F\224U\000\000\071\270\325F\224U\000\000x\243u\022\374\177\000\000\065\270\325F\224U\000\000\030iC\302.\177\000\000\366\257\273F\224U", '\000' <repeats 14 times>, "\b", '\000' <repeats 11 times>, "\060", '\000' <repeats 11 times>, "\002\000\000\000\000\000\000\000"...
[21:51:38.541]         xlogRestoreCmd = 0x559448ec08d8 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/025_stuck_on_old_timeline/data/t_025_stuck_on_old_timeline_primary_data/archives/00000003.history\" \"pg_wal/RECOVERYHISTORY\""
[21:51:38.541]         lastRestartPointFname = '0' <repeats 24 times>, "\000J\326F\224U\000\000\030\237u\022\374\177\000\000#J\326F\224U\000\000\030iC\302.\177\000\000\264\260\273F\224U", '\000' <repeats 14 times>, "\b", '\000' <repeats 11 times>, "\060", '\000' <repeats 11 times>, "\002", '\000' <repeats 99 times>...
[21:51:38.541]         rc = <optimized out>
[21:51:38.541]         stat_buf = {st_dev = 3472113003596310384, st_ino = 206966894640, st_nlink = 34184295202646376, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 8, st_size = 5922071200885860208, st_blksize = 6366210006019162949, st_blocks = 94094148194124, st_atim = {tv_sec = 94095330223704, tv_nsec = 94095331932225}, st_mtim = {tv_sec = 3747088985304184889, tv_nsec = 140720618184392}, st_ctim = {tv_sec = 1188411445, tv_nsec = 139838804420888}, __glibc_reserved = {8, 94095331955276, 140720618184472}}
[21:51:38.541]         restartSegNo = <optimized out>
[21:51:38.541]         restartRedoPtr = 94095330224940
[21:51:38.541]         restartTli = 0
[21:51:38.541]         __func__ = "RestoreArchivedFile"
[21:51:38.541] #6  0x000055944622a57d in existsTimeLineHistory (probeTLI=probeTLI@entry=3) at ../src/backend/access/transam/timeline.c:235
[21:51:38.541]         path = '\000' <repeats 16 times>, "\002\000\000\000\000\000\000\000\242J&F\224U\000\000@\250u\022\374\177\000\000\a\000\000\000\r\000\000\000\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\300\250u\022\374\177\000\000p\250u\022\374\177\000\000\001\000\000\000\000\000\000\000\004\000\000\t", '\000' <repeats 12 times>, "D\246u\022\374\177\000\000\030\262\341F\224U\000\000q3vG\224U\000\000\340\063vG\224U\000\000\240\037\000\000\377\377\002\000'\262\341F\224U\000\000\340\063vG\224U\000\000\002\000\000\000\000\000\000\000U\300\273F\224U\000\000f\246u\022\374\177\000\000d\246u\022\374\177", '\000' <repeats 26 times>...
[21:51:38.541]         histfname = "00000003.history", '\000' <repeats 47 times>
[21:51:38.541]         fd = <optimized out>
[21:51:38.541]         __func__ = "existsTimeLineHistory"
[21:51:38.541] #7  0x000055944622a668 in findNewestTimeLine (startTLI=startTLI@entry=2) at ../src/backend/access/transam/timeline.c:277
[21:51:38.541]         newestTLI = 2
[21:51:38.541]         probeTLI = 3
[21:51:38.541] #8  0x0000559446260b6e in rescanLatestTimeLine (replayTLI=replayTLI@entry=2, replayLSN=replayLSN@entry=67211504) at ../src/backend/access/transam/xlogrecovery.c:4113
[21:51:38.541]         newExpectedTLEs = <optimized out>
[21:51:38.541]         found = <optimized out>
[21:51:38.541]         cell = <optimized out>
[21:51:38.541]         newtarget = <optimized out>
[21:51:38.541]         oldtarget = 2
[21:51:38.541]         currentTle = 0x0
[21:51:38.541]         __func__ = "rescanLatestTimeLine"
[21:51:38.541] #9  0x0000559446265d1e in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=67211528, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=67211504, replayTLI=2, replayLSN=67211504, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3674
[21:51:38.541]         oldSource = XLOG_FROM_STREAM
[21:51:38.541]         startWalReceiver = false
[21:51:38.541]         last_fail_time = 767483261574486
[21:51:38.541]         now = <optimized out>
[21:51:38.541]         streaming_reply_sent = true
[21:51:38.541]         __func__ = "WaitForWALToBecomeAvailable"
[21:51:38.541] #10 0x0000559446266754 in XLogPageRead (xlogreader=0x559448e6deb0, targetPagePtr=67207168, reqLen=4360, targetRecPtr=67211504, readBuf=0x559448ebd0a8 "\025\321\006") at ../src/backend/access/transam/xlogrecovery.c:3351
[21:51:38.541]         private = 0x559448e6de90
[21:51:38.541]         emode = 15
[21:51:38.541]         targetPageOff = 98304
[21:51:38.541]         targetSegNo = 4
[21:51:38.541]         r = <optimized out>
[21:51:38.541]         __func__ = "XLogPageRead"
[21:51:38.541] #11 0x000055944626ebaf in ReadPageInternal (state=state@entry=0x559448e6deb0, pageptr=pageptr@entry=67207168, reqLen=reqLen@entry=4360) at ../src/backend/access/transam/xlogreader.c:1060
[21:51:38.541]         readLen = <optimized out>
[21:51:38.541]         targetPageOff = 98304
[21:51:38.541]         targetSegNo = 4
[21:51:38.541]         hdr = <optimized out>
[21:51:38.541] #12 0x000055944627080a in XLogDecodeNextRecord (state=state@entry=0x559448e6deb0, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:597
[21:51:38.541]         RecPtr = 67211504
[21:51:38.541]         record = <optimized out>
[21:51:38.541]         targetPagePtr = 67207168
[21:51:38.541]         randAccess = false
[21:51:38.541]         len = <optimized out>
[21:51:38.541]         total_len = <optimized out>
[21:51:38.541]         targetRecOff = 4336
[21:51:38.541]         pageHeaderSize = <optimized out>
[21:51:38.541]         assembled = false
[21:51:38.541]         gotheader = <optimized out>
[21:51:38.541]         readOff = <optimized out>
[21:51:38.541]         decoded = 0x0
[21:51:38.541]         errormsg = 0x559446e09581 "LOG"
[21:51:38.541] #13 0x00005594462713ac in XLogReadAhead (state=0x559448e6deb0, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:973
[21:51:38.541]         result = <optimized out>
[21:51:38.541] #14 0x000055944625bcee in XLogPrefetcherNextBlock (pgsr_private=94095366614240, lsn=0x559448ed2bd8) at ../src/backend/access/transam/xlogprefetcher.c:492
[21:51:38.541]         nonblocking = <optimized out>
[21:51:38.541]         record = <optimized out>
[21:51:38.541]         prefetcher = 0x559448e6ece0
[21:51:38.541]         reader = 0x559448e6deb0
[21:51:38.541]         replaying_lsn = 67211096
[21:51:38.541]         __func__ = "XLogPrefetcherNextBlock"
[21:51:38.541] #15 0x00005594462600b7 in lrq_prefetch (lrq=<optimized out>) at ../src/backend/access/transam/xlogprefetcher.c:252
[21:51:38.541] No locals.
[21:51:38.541] #16 lrq_complete_lsn (lsn=0, lrq=0x559448ed2958) at ../src/backend/access/transam/xlogprefetcher.c:290
[21:51:38.541] No locals.
[21:51:38.541] #17 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x559448e6ece0, errmsg=errmsg@entry=0x7ffc1275ebd8) at ../src/backend/access/transam/xlogprefetcher.c:1037
[21:51:38.541]         record = <optimized out>
[21:51:38.541]         replayed_up_to = 0
[21:51:38.541] #18 0x0000559446266f81 in ReadRecord (xlogprefetcher=0x559448e6ece0, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=2) at ../src/backend/access/transam/xlogrecovery.c:3151
[21:51:38.541]         errormsg = 0x559448e6e6c0 ""
[21:51:38.541]         record = <optimized out>
[21:51:38.541]         xlogreader = 0x559448e6deb0
[21:51:38.541]         private = <optimized out>
[21:51:38.541]         __func__ = "ReadRecord"
[21:51:38.541] #19 0x000055944626a118 in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1832
[21:51:38.541]         xtime = <optimized out>
[21:51:38.541]         ru0 = {tv = {tv_sec = 1714168061, tv_usec = 29776}, ru = {ru_utime = {tv_sec = 0, tv_usec = 1979}, ru_stime = {tv_sec = 0, tv_usec = 0}, {ru_maxrss = 16208, __ru_maxrss_word = 16208}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 404, __ru_minflt_word = 404}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 11, __ru_nvcsw_word = 11}, {ru_nivcsw = 1, __ru_nivcsw_word = 1}}}
[21:51:38.541]         record = 0x7f2ec1d09e50
[21:51:38.560]         reachedRecoveryTarget = false
[21:51:38.560]         replayTLI = 2
[21:51:38.560]         __func__ = "PerformWalRecovery"
[21:51:38.560] #20 0x0000559446253b3b in StartupXLOG () at ../src/backend/access/transam/xlog.c:5806
[21:51:38.560]         Insert = <optimized out>
[21:51:38.560]         checkPoint = {redo = 33554472, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 738}, nextOid = 14202, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1714168059, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 738}
[21:51:38.560]         wasShutdown = false
[21:51:38.560]         didCrash = <optimized out>
[21:51:38.560]         haveTblspcMap = false
[21:51:38.560]         haveBackupLabel = true
[21:51:38.560]         EndOfLog = <optimized out>
[21:51:38.560]         EndOfLogTLI = <optimized out>
[21:51:38.560]         newTLI = <optimized out>
[21:51:38.560]         performedWalRecovery = <optimized out>
[21:51:38.560]         endOfRecoveryInfo = <optimized out>
[21:51:38.560]         abortedRecPtr = <optimized out>
[21:51:38.560]         missingContrecPtr = <optimized out>
[21:51:38.560]         oldestActiveXID = <optimized out>
[21:51:38.560]         promoted = false
[21:51:38.560]         __func__ = "StartupXLOG"
[21:51:38.560] #21 0x000055944671970e in StartupProcessMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at ../src/backend/postmaster/startup.c:258
[21:51:38.560] No locals.
[21:51:38.560] #22 0x0000559446710d2a in postmaster_child_launch (child_type=child_type@entry=B_STARTUP, startup_data=startup_data@entry=0x0, startup_data_len=startup_data_len@entry=0, client_sock=client_sock@entry=0x0) at ../src/backend/postmaster/launch_backend.c:265
[21:51:38.560]         pid = <optimized out>
[21:51:38.560] #23 0x0000559446712da3 in StartChildProcess (type=type@entry=B_STARTUP) at ../src/backend/postmaster/postmaster.c:3928
[21:51:38.560]         pid = <optimized out>
[21:51:38.560]         __func__ = "StartChildProcess"
[21:51:38.560] #24 0x0000559446718e6d in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x559448e6cd70) at ../src/backend/postmaster/postmaster.c:1364
[21:51:38.560]         opt = <optimized out>
[21:51:38.560]         status = <optimized out>
[21:51:38.560]         userDoption = <optimized out>
[21:51:38.560]         listen_addr_saved = false
[21:51:38.560]         output_config_variable = <optimized out>
[21:51:38.560]         __func__ = "PostmasterMain"
[21:51:38.560] #25 0x000055944653981d in main (argc=4, argv=0x559448e6cd70) at ../src/backend/main/main.c:197
[21:51:38.560]         do_check_root = <optimized out>
[21:51:38.560] 
[21:51:38.560] Thread 1 (LWP 20076):
[21:51:38.560] #0  0x00007f2ec4a9ffc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6
[21:51:38.560] No symbol table info available.
[21:51:38.560] #1  0x00007f2ec4b5113a in ?? () from /lib/x86_64-linux-gnu/libc.so.6
[21:51:38.560] No symbol table info available.
[21:51:38.560] #2  0x00007f2ec4b62a6f in clone () from /lib/x86_64-linux-gnu/libc.so.6
[21:51:38.560] No symbol table info available.