[10:09:26.050] src/tools/ci/cores_backtrace.sh linux /tmp/cores [10:09:26.828] dumping /tmp/cores/postgres-3-18491.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres [10:09:28.590] [New LWP 18491] [10:09:28.590] [New LWP 18436] [10:09:28.590] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [10:09:29.387] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [10:09:30.191] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [10:09:31.445] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [10:09:33.924] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [10:09:33.924] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo... [10:09:35.861] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [10:09:37.124] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [10:09:38.766] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [10:09:39.875] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [10:09:40.659] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [10:09:41.935] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [10:09:43.583] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [10:09:45.026] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [10:09:45.981] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [10:09:47.576] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [10:09:49.030] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [10:09:49.512] [Thread debugging using libthread_db enabled] [10:09:49.512] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [10:09:50.621] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [10:09:52.070] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [10:09:53.159] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [10:09:53.801] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [10:09:54.585] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [10:09:56.935] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [10:09:57.907] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [10:09:58.688] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [10:09:59.639] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [10:10:00.594] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [10:10:02.212] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [10:10:03.295] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [10:10:04.076] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [10:10:05.487] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [10:10:06.128] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [10:10:07.727] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [10:10:08.510] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [10:10:09.764] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [10:10:10.566] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [10:10:11.678] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [10:10:12.619] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [10:10:13.887] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [10:10:14.518] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [10:10:15.476] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [10:10:16.272] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [10:10:16.806] Core was generated by `postgres: standby2: startup waiting for 000000010000000000000003 '. [10:10:16.806] Program terminated with signal SIGQUIT, Quit. [10:10:16.806] #0 0x00007f7e5cabefc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [10:10:16.839] [Current thread is 1 (LWP 18491)] [10:10:16.980] [10:10:16.980] Thread 2 (Thread 0x7f7e5a455a40 (LWP 18436)): [10:10:16.980] #0 0x00007f7e5cb81a61 in clone () from /lib/x86_64-linux-gnu/libc.so.6 [10:10:16.980] No symbol table info available. [10:10:16.980] #1 0x00007f7e5cb6fe5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [10:10:16.980] No symbol table info available. [10:10:16.980] #2 0x00007f7e5cb70497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [10:10:16.980] No symbol table info available. [10:10:16.980] #3 0x00007f7e5cb6fcdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6 [10:10:16.980] No symbol table info available. [10:10:16.980] #4 0x00007f7e5cacba69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [10:10:16.980] No symbol table info available. [10:10:16.980] #5 0x000055f3e5fc72e9 in RestoreArchivedFile (path=path@entry=0x7ffe96a59f50 "\001", xlogfname=xlogfname@entry=0x7ffe96a59f10 "00000002.history", recovername=recovername@entry=0x55f3e6ad5b50 "RECOVERYHISTORY", expectedSize=expectedSize@entry=0, cleanupEnabled=cleanupEnabled@entry=false) at ../src/backend/access/transam/xlogarchive.c:177 [10:10:16.980] xlogpath = "pg_wal/RECOVERYHISTORY\000\000\b", '\000' , "\231\265\222\346\363U\000\062", '\000' , "\b\000\000\000\000\000\000\000\231Ǭ\346\363U\000\000\070\236\245\226\376\177\000\000\225Ǭ\346\363U\000\000Ű\222\346\363U\000\000\241Ǭ\346\363U\000\000\231Ǭ\346\363U\000\000\070\236\245\226\376\177\000\000\225Ǭ\346\363U\000\000\030YEZ~\177\000\000c\272\222\346\363U", '\000' , "\b", '\000' , "\060", '\000' , "\002\000\000\000\000\000\000\000~\000\000\000"... [10:10:16.980] xlogRestoreCmd = 0x55f3e7f035c8 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/020_archive_status/data/t_020_archive_status_primary_data/archives/00000002.history\" \"pg_wal/RECOVERYHISTORY\"" [10:10:16.980] lastRestartPointFname = '0' , "\000Ò\346\363U", '\000' , "\220\231\245\226\376\177\000\000c\272\222\346\363U", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' ... [10:10:16.980] rc = [10:10:16.980] stat_buf = {st_dev = 32768, st_ino = 207871778576, st_nlink = 929105816, st_mode = 1713348368, st_uid = 0, st_gid = 765105893, __pad0 = 0, st_rdev = 1713348368, st_size = 5922071200885860208, st_blksize = 6366210006019162949, st_blocks = 4673356, st_atim = {tv_sec = 0, tv_nsec = 3472113003596310384}, st_mtim = {tv_sec = 3688782553318174768, tv_nsec = 3472328296227680304}, st_ctim = {tv_sec = 808464432, tv_nsec = 0}, __glibc_reserved = {8, 94506035520300, 140731425855960}} [10:10:16.980] restartSegNo = [10:10:16.980] restartRedoPtr = 94506033788313 [10:10:16.980] restartTli = 0 [10:10:16.980] __func__ = "RestoreArchivedFile" [10:10:16.980] #6 0x000055f3e5f9bcb6 in existsTimeLineHistory (probeTLI=probeTLI@entry=2) at ../src/backend/access/transam/timeline.c:235 [10:10:16.980] path = "\001", '\000' , "\001\000\000\000\000\000\000\000\353c\375\345\363U\000\000 \017\353\347\363U\000\000\377\377\377\377\r\000\000\000pg_wal/00000001", '0' , "3\000\000\000\000\000\000\000\000\000h\310\305\\~\177\000\000`\243\245\226\376\177\000\000\212\026A^~\177\000\000ȯK\347\363U", '\000' , " \000\000\000\000\000\000\230\375\357\347\363U\000\000\006\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000\060", '\000' , "\240"... [10:10:16.980] histfname = "00000002.history", '\000' , "x]\361\347\363U\000\000\001\000\000\000\000\000\000" [10:10:16.980] fd = [10:10:16.980] __func__ = "existsTimeLineHistory" [10:10:16.980] #7 0x000055f3e5f9bda1 in findNewestTimeLine (startTLI=startTLI@entry=1) at ../src/backend/access/transam/timeline.c:277 [10:10:16.980] newestTLI = 1 [10:10:16.980] probeTLI = 2 [10:10:16.980] #8 0x000055f3e5fd22b4 in rescanLatestTimeLine (replayTLI=replayTLI@entry=1, replayLSN=replayLSN@entry=50331648) at ../src/backend/access/transam/xlogrecovery.c:4113 [10:10:16.980] newExpectedTLEs = [10:10:16.980] found = [10:10:16.980] cell = [10:10:16.980] newtarget = [10:10:16.980] oldtarget = 1 [10:10:16.980] currentTle = 0x0 [10:10:16.980] __func__ = "rescanLatestTimeLine" [10:10:16.980] #9 0x000055f3e5fd7464 in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=50331672, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=50331648, replayTLI=1, replayLSN=50331648, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3674 [10:10:16.980] oldSource = XLOG_FROM_STREAM [10:10:16.980] startWalReceiver = false [10:10:16.980] last_fail_time = 766663569052714 [10:10:16.980] now = [10:10:16.980] streaming_reply_sent = false [10:10:16.980] __func__ = "WaitForWALToBecomeAvailable" [10:10:16.980] #10 0x000055f3e5fd7e9a in XLogPageRead (xlogreader=0x55f3e7eb0f40, targetPagePtr=50331648, reqLen=24, targetRecPtr=50331648, readBuf=0x55f3e7effd98 "\025\321\005") at ../src/backend/access/transam/xlogrecovery.c:3351 [10:10:16.980] private = 0x55f3e7eb0f20 [10:10:16.980] emode = 15 [10:10:16.980] targetPageOff = 0 [10:10:16.980] targetSegNo = 3 [10:10:16.980] r = [10:10:16.980] __func__ = "XLogPageRead" [10:10:16.980] #11 0x000055f3e5fe02f5 in ReadPageInternal (state=state@entry=0x55f3e7eb0f40, pageptr=pageptr@entry=50331648, reqLen=reqLen@entry=24) at ../src/backend/access/transam/xlogreader.c:1060 [10:10:16.980] readLen = [10:10:16.980] targetPageOff = 0 [10:10:16.980] targetSegNo = 3 [10:10:16.980] hdr = [10:10:16.980] #12 0x000055f3e5fe1f50 in XLogDecodeNextRecord (state=state@entry=0x55f3e7eb0f40, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:597 [10:10:16.980] RecPtr = 50331648 [10:10:16.980] record = [10:10:16.980] targetPagePtr = 50331648 [10:10:16.980] randAccess = false [10:10:16.980] len = [10:10:16.980] total_len = [10:10:16.980] targetRecOff = 0 [10:10:16.980] pageHeaderSize = [10:10:16.980] assembled = false [10:10:16.980] gotheader = [10:10:16.980] readOff = [10:10:16.980] decoded = 0x0 [10:10:16.980] errormsg = 0x55f3e7eb0f40 "5|\375\345\363U" [10:10:16.980] #13 0x000055f3e5fe2af2 in XLogReadAhead (state=0x55f3e7eb0f40, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:973 [10:10:16.980] result = [10:10:16.980] #14 0x000055f3e5fcd434 in XLogPrefetcherNextBlock (pgsr_private=94506056359280, lsn=0x55f3e7f15748) at ../src/backend/access/transam/xlogprefetcher.c:492 [10:10:16.980] nonblocking = [10:10:16.980] record = [10:10:16.980] prefetcher = 0x55f3e7eb1d70 [10:10:16.980] reader = 0x55f3e7eb0f40 [10:10:16.980] replaying_lsn = 33563904 [10:10:16.980] __func__ = "XLogPrefetcherNextBlock" [10:10:16.980] #15 0x000055f3e5fd17fd in lrq_prefetch (lrq=) at ../src/backend/access/transam/xlogprefetcher.c:252 [10:10:16.980] No locals. [10:10:16.980] #16 lrq_complete_lsn (lsn=50331648, lrq=0x55f3e7f15648) at ../src/backend/access/transam/xlogprefetcher.c:290 [10:10:16.980] No locals. [10:10:16.980] #17 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x55f3e7eb1d70, errmsg=errmsg@entry=0x7ffe96a5e698) at ../src/backend/access/transam/xlogprefetcher.c:1037 [10:10:16.980] record = [10:10:16.980] replayed_up_to = 50331648 [10:10:16.980] #18 0x000055f3e5fd86c7 in ReadRecord (xlogprefetcher=0x55f3e7eb1d70, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=1) at ../src/backend/access/transam/xlogrecovery.c:3151 [10:10:16.980] errormsg = 0x55f3e64d81df "AUATUSH\203\354\bH\205\377\017\204:\001" [10:10:16.980] record = [10:10:16.980] xlogreader = 0x55f3e7eb0f40 [10:10:16.980] private = [10:10:16.980] __func__ = "ReadRecord" [10:10:16.980] #19 0x000055f3e5fdb85e in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1832 [10:10:16.980] xtime = [10:10:16.980] ru0 = {tv = {tv_sec = 1713348368, tv_usec = 931192}, ru = {ru_utime = {tv_sec = 0, tv_usec = 1389}, ru_stime = {tv_sec = 0, tv_usec = 0}, {ru_maxrss = 15948, __ru_maxrss_word = 15948}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 398, __ru_minflt_word = 398}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 2, __ru_nvcsw_word = 2}, {ru_nivcsw = 1, __ru_nivcsw_word = 1}}} [10:10:16.980] record = 0x7f7e59d11690 [10:10:16.980] reachedRecoveryTarget = false [10:10:16.980] replayTLI = 1 [10:10:16.980] __func__ = "PerformWalRecovery" [10:10:17.001] #20 0x000055f3e5fc5281 in StartupXLOG () at ../src/backend/access/transam/xlog.c:5806 [10:10:17.001] Insert = [10:10:17.001] checkPoint = {redo = 33554472, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 739}, nextOid = 24576, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1713348366, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 739} [10:10:17.001] wasShutdown = false [10:10:17.001] didCrash = [10:10:17.001] haveTblspcMap = false [10:10:17.001] haveBackupLabel = false [10:10:17.001] EndOfLog = [10:10:17.001] EndOfLogTLI = [10:10:17.001] newTLI = [10:10:17.001] performedWalRecovery = [10:10:17.001] endOfRecoveryInfo = [10:10:17.001] abortedRecPtr = [10:10:17.001] missingContrecPtr = [10:10:17.001] oldestActiveXID = [10:10:17.001] promoted = false [10:10:17.001] __func__ = "StartupXLOG" [10:10:17.001] #21 0x000055f3e648a288 in StartupProcessMain (startup_data=, startup_data_len=) at ../src/backend/postmaster/startup.c:258 [10:10:17.001] No locals. [10:10:17.001] #22 0x000055f3e64818a4 in postmaster_child_launch (child_type=child_type@entry=B_STARTUP, startup_data=startup_data@entry=0x0, startup_data_len=startup_data_len@entry=0, client_sock=client_sock@entry=0x0) at ../src/backend/postmaster/launch_backend.c:265 [10:10:17.001] pid = [10:10:17.001] #23 0x000055f3e648391d in StartChildProcess (type=type@entry=B_STARTUP) at ../src/backend/postmaster/postmaster.c:3928 [10:10:17.001] pid = [10:10:17.001] __func__ = "StartChildProcess" [10:10:17.001] #24 0x000055f3e64899e7 in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x55f3e7eafe10) at ../src/backend/postmaster/postmaster.c:1364 [10:10:17.001] opt = [10:10:17.001] status = [10:10:17.001] userDoption = [10:10:17.001] listen_addr_saved = false [10:10:17.001] output_config_variable = [10:10:17.001] __func__ = "PostmasterMain" [10:10:17.001] #25 0x000055f3e62aa443 in main (argc=4, argv=0x55f3e7eafe10) at ../src/backend/main/main.c:197 [10:10:17.001] do_check_root = [10:10:17.001] [10:10:17.001] Thread 1 (LWP 18491): [10:10:17.001] #0 0x00007f7e5cabefc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [10:10:17.001] No symbol table info available. [10:10:17.001] #1 0x00007f7e5cb7013a in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [10:10:17.001] No symbol table info available. [10:10:17.001] #2 0x00007f7e5cb81a6f in clone () from /lib/x86_64-linux-gnu/libc.so.6 [10:10:17.001] No symbol table info available.