[05:09:56.226] src/tools/ci/cores_backtrace.sh linux /tmp/cores [05:09:56.921] dumping /tmp/cores/postgres-3-20994.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres [05:09:58.578] [New LWP 20994] [05:09:58.578] [New LWP 19875] [05:09:58.578] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [05:09:59.340] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [05:10:00.098] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [05:10:01.318] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [05:10:03.669] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [05:10:03.669] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo... [05:10:05.536] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [05:10:06.740] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [05:10:08.310] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [05:10:09.367] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [05:10:10.131] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [05:10:11.354] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [05:10:12.959] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [05:10:14.362] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [05:10:15.268] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [05:10:16.799] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [05:10:18.182] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [05:10:18.652] [Thread debugging using libthread_db enabled] [05:10:18.652] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [05:10:19.713] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [05:10:21.100] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [05:10:22.161] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [05:10:22.775] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [05:10:23.539] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [05:10:25.811] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [05:10:26.723] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [05:10:27.497] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [05:10:28.410] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [05:10:29.321] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [05:10:30.909] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [05:10:31.965] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [05:10:32.729] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [05:10:34.112] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [05:10:34.725] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [05:10:36.257] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [05:10:37.020] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [05:10:38.238] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [05:10:39.002] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [05:10:40.066] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [05:10:40.981] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [05:10:42.199] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [05:10:42.814] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [05:10:43.727] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [05:10:44.488] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [05:10:44.975] Core was generated by `postgres: standby: startup waiting for 000000010000000000000005 '. [05:10:44.975] Program terminated with signal SIGQUIT, Quit. [05:10:44.975] #0 0x00007f09a08f3fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [05:10:45.002] [Current thread is 1 (LWP 20994)] [05:10:45.111] [05:10:45.111] Thread 2 (Thread 0x7f099e28aa40 (LWP 19875)): [05:10:45.111] #0 0x00007f09a09b6a61 in clone () from /lib/x86_64-linux-gnu/libc.so.6 [05:10:45.111] No symbol table info available. [05:10:45.111] #1 0x00007f09a09a4e5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [05:10:45.111] No symbol table info available. [05:10:45.111] #2 0x00007f09a09a5497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [05:10:45.111] No symbol table info available. [05:10:45.111] #3 0x00007f09a09a4cdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6 [05:10:45.111] No symbol table info available. [05:10:45.111] #4 0x00007f09a0900a69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [05:10:45.111] No symbol table info available. [05:10:45.111] #5 0x0000558759287bb0 in RestoreArchivedFile (path=path@entry=0x7ffd4510bd40 "\001", xlogfname=xlogfname@entry=0x7ffd4510bd00 "00000002.history", recovername=recovername@entry=0x558759d73d10 "RECOVERYHISTORY", expectedSize=expectedSize@entry=0, cleanupEnabled=cleanupEnabled@entry=false) at ../src/backend/access/transam/xlogarchive.c:177 [05:10:45.111] xlogpath = "pg_wal/RECOVERYHISTORY\000\000\b", '\000' , "\236\062\275Y\207U\000\062", '\000' , "\b\000\000\000\000\000\000\000Y\252\326Y\207U\000\000(\274\020E\375\177\000\000U\252\326Y\207U\000\000\312-\275Y\207U\000\000a\252\326Y\207U\000\000Y\252\326Y\207U\000\000(\274\020E\375\177\000\000U\252\326Y\207U\000\000\030\251(\236\t\177\000\000h7\275Y\207U", '\000' , "\b", '\000' , "\060", '\000' , "\002\000\000\000\000\000\000\000"... [05:10:45.111] xlogRestoreCmd = 0x55875c57a628 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/020_archive_status/data/t_020_archive_status_primary_data/archives/00000002.history\" \"pg_wal/RECOVERYHISTORY\"" [05:10:45.111] lastRestartPointFname = '0' , "\000@\275Y\207U", '\000' , "\200\267\020E\375\177\000\000h7\275Y\207U", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' ... [05:10:45.111] rc = [05:10:45.111] stat_buf = {st_dev = 32768, st_ino = 207870032635, st_nlink = 924042400, st_mode = 1711602427, st_uid = 0, st_gid = 772042479, __pad0 = 0, st_rdev = 1711602427, st_size = 5922071200885860208, st_blksize = 6366210006019162949, st_blocks = 4673356, st_atim = {tv_sec = 0, tv_nsec = 3472113003596310384}, st_mtim = {tv_sec = 3832897741394030640, tv_nsec = 3472328296227680304}, st_ctim = {tv_sec = 808464432, tv_nsec = 0}, __glibc_reserved = {8, 94039816209628, 140725762176968}} [05:10:45.111] restartSegNo = [05:10:45.111] restartRedoPtr = 94039814517406 [05:10:45.111] restartTli = 0 [05:10:45.111] __func__ = "RestoreArchivedFile" [05:10:45.111] #6 0x000055875925dfbe in existsTimeLineHistory (probeTLI=probeTLI@entry=2) at ../src/backend/access/transam/timeline.c:235 [05:10:45.111] path = "\001", '\000' , "\001\000\000\000\000\000\000\000\262l)Y\207U\000\000<,\327Y\207U\000\000\377\377\377\377\r\000\000\000pg_wal/00000001", '0' , "5\000\000\000\000\000\000\000\000\000b\025uZ\207U\000\000\340\025uZ\207U\000\000\364\276\020E\375\177\000\000\070~\342Y\207U\000\000q\025uZ\207U\000\000\340\025uZ\207U\000\000\024\277\020E\375\177\000\000G~\342Y\207U\000\000\340\025uZ\207U\000\000\002\000\000\000\000\000\000\000\307G\275Y\207U\000\000\026\277\020E\375\177\000\000\024\277\020E\375\177", '\000' ... [05:10:45.111] histfname = "00000002.history\000\000\000\000\000\000\000\000XvR\\\207U\000\000\005\000\000\000\002\000\000\000\004\000\000\t\000\000\000\000\330\315X\\\207U\000\000\001\000\000\000\000\000\000" [05:10:45.111] fd = [05:10:45.111] __func__ = "existsTimeLineHistory" [05:10:45.111] #7 0x000055875925e0a9 in findNewestTimeLine (startTLI=startTLI@entry=1) at ../src/backend/access/transam/timeline.c:277 [05:10:45.111] newestTLI = 1 [05:10:45.111] probeTLI = 2 [05:10:45.111] #8 0x0000558759292b7b in rescanLatestTimeLine (replayTLI=replayTLI@entry=1, replayLSN=replayLSN@entry=83886080) at ../src/backend/access/transam/xlogrecovery.c:4113 [05:10:45.111] newExpectedTLEs = [05:10:45.111] found = [05:10:45.111] cell = [05:10:45.111] newtarget = [05:10:45.111] oldtarget = 1 [05:10:45.111] currentTle = 0x0 [05:10:45.111] __func__ = "rescanLatestTimeLine" [05:10:45.111] #9 0x0000558759297d2b in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=83886104, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=83886080, replayTLI=1, replayLSN=83886080, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3674 [05:10:45.111] oldSource = XLOG_FROM_STREAM [05:10:45.111] startWalReceiver = false [05:10:45.111] last_fail_time = 764917631533048 [05:10:45.111] now = [05:10:45.111] streaming_reply_sent = false [05:10:45.111] __func__ = "WaitForWALToBecomeAvailable" [05:10:45.111] #10 0x0000558759298761 in XLogPageRead (xlogreader=0x55875c527f80, targetPagePtr=83886080, reqLen=24, targetRecPtr=83886080, readBuf=0x55875c576df8 "\025\321\002") at ../src/backend/access/transam/xlogrecovery.c:3351 [05:10:45.111] private = 0x55875c527f60 [05:10:45.111] emode = 15 [05:10:45.111] targetPageOff = 0 [05:10:45.111] targetSegNo = 5 [05:10:45.111] r = [05:10:45.111] __func__ = "XLogPageRead" [05:10:45.111] #11 0x00005587592a0bbc in ReadPageInternal (state=state@entry=0x55875c527f80, pageptr=pageptr@entry=83886080, reqLen=reqLen@entry=24) at ../src/backend/access/transam/xlogreader.c:1060 [05:10:45.111] readLen = [05:10:45.111] targetPageOff = 0 [05:10:45.111] targetSegNo = 5 [05:10:45.111] hdr = [05:10:45.111] #12 0x00005587592a2817 in XLogDecodeNextRecord (state=state@entry=0x55875c527f80, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:597 [05:10:45.111] RecPtr = 83886080 [05:10:45.111] record = [05:10:45.111] targetPagePtr = 83886080 [05:10:45.111] randAccess = false [05:10:45.111] len = [05:10:45.111] total_len = [05:10:45.111] targetRecOff = 0 [05:10:45.111] pageHeaderSize = [05:10:45.111] assembled = false [05:10:45.111] gotheader = [05:10:45.111] readOff = [05:10:45.111] decoded = 0x0 [05:10:45.111] errormsg = 0x55875c527f80 "\374\204)Y\207U" [05:10:45.111] #13 0x00005587592a33b9 in XLogReadAhead (state=0x55875c527f80, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:973 [05:10:45.111] result = [05:10:45.111] #14 0x000055875928dcfb in XLogPrefetcherNextBlock (pgsr_private=94039857860016, lsn=0x55875c58c7a8) at ../src/backend/access/transam/xlogprefetcher.c:492 [05:10:45.111] nonblocking = [05:10:45.111] record = [05:10:45.111] prefetcher = 0x55875c528db0 [05:10:45.111] reader = 0x55875c527f80 [05:10:45.111] replaying_lsn = 67109152 [05:10:45.111] __func__ = "XLogPrefetcherNextBlock" [05:10:45.111] #15 0x00005587592920c4 in lrq_prefetch (lrq=) at ../src/backend/access/transam/xlogprefetcher.c:252 [05:10:45.111] No locals. [05:10:45.111] #16 lrq_complete_lsn (lsn=83886080, lrq=0x55875c58c6a8) at ../src/backend/access/transam/xlogprefetcher.c:290 [05:10:45.111] No locals. [05:10:45.111] #17 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x55875c528db0, errmsg=errmsg@entry=0x7ffd45110488) at ../src/backend/access/transam/xlogprefetcher.c:1037 [05:10:45.111] record = [05:10:45.111] replayed_up_to = 83886080 [05:10:45.111] #18 0x0000558759298f8e in ReadRecord (xlogprefetcher=0x55875c528db0, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=1) at ../src/backend/access/transam/xlogrecovery.c:3151 [05:10:45.111] errormsg = 0x55875978b92f "AUATUSH\203\354\bH\205\377\017\204:\001" [05:10:45.111] record = [05:10:45.111] xlogreader = 0x55875c527f80 [05:10:45.111] private = [05:10:45.111] __func__ = "ReadRecord" [05:10:45.111] #19 0x000055875929c125 in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1832 [05:10:45.111] xtime = [05:10:45.111] ru0 = {tv = {tv_sec = 1711602427, tv_usec = 927515}, ru = {ru_utime = {tv_sec = 0, tv_usec = 1385}, ru_stime = {tv_sec = 0, tv_usec = 0}, {ru_maxrss = 16256, __ru_maxrss_word = 16256}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 405, __ru_minflt_word = 405}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 2, __ru_nvcsw_word = 2}, {ru_nivcsw = 1, __ru_nivcsw_word = 1}}} [05:10:45.111] record = 0x7f099db441f0 [05:10:45.130] reachedRecoveryTarget = false [05:10:45.130] replayTLI = 1 [05:10:45.130] __func__ = "PerformWalRecovery" [05:10:45.130] #20 0x0000558759285dd2 in StartupXLOG () at ../src/backend/access/transam/xlog.c:5723 [05:10:45.130] Insert = [05:10:45.130] checkPoint = {redo = 33554472, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 739}, nextOid = 24576, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1711602426, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 739} [05:10:45.130] wasShutdown = false [05:10:45.130] didCrash = [05:10:45.130] haveTblspcMap = false [05:10:45.130] haveBackupLabel = false [05:10:45.130] EndOfLog = [05:10:45.130] EndOfLogTLI = [05:10:45.130] newTLI = [05:10:45.130] performedWalRecovery = [05:10:45.130] endOfRecoveryInfo = [05:10:45.130] abortedRecPtr = [05:10:45.130] missingContrecPtr = [05:10:45.130] oldestActiveXID = [05:10:45.130] promoted = false [05:10:45.130] __func__ = "StartupXLOG" [05:10:45.130] #21 0x000055875973d679 in StartupProcessMain (startup_data=, startup_data_len=) at ../src/backend/postmaster/startup.c:258 [05:10:45.130] No locals. [05:10:45.130] #22 0x0000558759734e43 in postmaster_child_launch (child_type=child_type@entry=B_STARTUP, startup_data=startup_data@entry=0x0, startup_data_len=startup_data_len@entry=0, client_sock=client_sock@entry=0x0) at ../src/backend/postmaster/launch_backend.c:265 [05:10:45.130] pid = [05:10:45.130] #23 0x0000558759736d0e in StartChildProcess (type=type@entry=B_STARTUP) at ../src/backend/postmaster/postmaster.c:3928 [05:10:45.130] pid = [05:10:45.130] __func__ = "StartChildProcess" [05:10:45.130] #24 0x000055875973cdd8 in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x55875c526e50) at ../src/backend/postmaster/postmaster.c:1364 [05:10:45.130] opt = [05:10:45.130] status = [05:10:45.130] userDoption = [05:10:45.130] listen_addr_saved = false [05:10:45.130] output_config_variable = [05:10:45.130] __func__ = "PostmasterMain" [05:10:45.130] #25 0x0000558759565581 in main (argc=4, argv=0x55875c526e50) at ../src/backend/main/main.c:197 [05:10:45.130] do_check_root = [05:10:45.130] [05:10:45.130] Thread 1 (LWP 20994): [05:10:45.130] #0 0x00007f09a08f3fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [05:10:45.130] No symbol table info available. [05:10:45.130] #1 0x00007f09a09a513a in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [05:10:45.130] No symbol table info available. [05:10:45.130] #2 0x00007f09a09b6a6f in clone () from /lib/x86_64-linux-gnu/libc.so.6 [05:10:45.130] No symbol table info available.