[11:44:08.304] src/tools/ci/cores_backtrace.sh linux /tmp/cores [11:44:09.007] dumping /tmp/cores/postgres-3-31776.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres [11:44:10.683] [New LWP 31776] [11:44:10.683] [New LWP 30677] [11:44:10.683] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [11:44:11.444] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [11:44:12.205] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [11:44:13.419] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [11:44:15.784] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [11:44:15.784] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo... [11:44:17.656] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [11:44:18.867] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [11:44:20.443] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [11:44:21.502] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [11:44:22.266] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [11:44:23.495] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [11:44:25.110] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [11:44:26.511] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [11:44:27.420] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [11:44:28.953] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [11:44:30.338] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [11:44:30.808] [Thread debugging using libthread_db enabled] [11:44:30.808] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [11:44:31.874] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [11:44:33.263] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [11:44:34.327] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [11:44:34.943] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [11:44:35.707] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [11:44:37.989] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [11:44:38.903] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [11:44:39.677] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [11:44:40.590] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [11:44:41.504] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [11:44:43.081] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [11:44:44.139] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [11:44:44.904] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [11:44:46.285] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [11:44:46.899] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [11:44:48.426] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [11:44:49.189] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [11:44:50.410] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [11:44:51.175] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [11:44:52.246] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [11:44:53.167] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [11:44:54.386] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [11:44:55.002] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [11:44:55.918] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [11:44:56.681] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [11:44:57.183] Core was generated by `postgres: standby: startup waiting for 000000010000000000000005 '. [11:44:57.183] Program terminated with signal SIGQUIT, Quit. [11:44:57.183] #0 0x00007fe8e48f2fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [11:44:57.211] [Current thread is 1 (LWP 31776)] [11:44:57.346] [11:44:57.346] Thread 2 (Thread 0x7fe8e2289a40 (LWP 30677)): [11:44:57.346] #0 0x00007fe8e49b5a61 in clone () from /lib/x86_64-linux-gnu/libc.so.6 [11:44:57.346] No symbol table info available. [11:44:57.346] #1 0x00007fe8e49a3e5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [11:44:57.346] No symbol table info available. [11:44:57.346] #2 0x00007fe8e49a4497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [11:44:57.346] No symbol table info available. [11:44:57.346] #3 0x00007fe8e49a3cdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6 [11:44:57.346] No symbol table info available. [11:44:57.346] #4 0x00007fe8e48ffa69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [11:44:57.346] No symbol table info available. [11:44:57.346] #5 0x0000561727c15e5f in RestoreArchivedFile (path=path@entry=0x7ffd21be7fb0 "P\203\276!\375\177", xlogfname=xlogfname@entry=0x7ffd21be8400 "00000001", '0' , "5", recovername=recovername@entry=0x561728718fbc "RECOVERYXLOG", expectedSize=16777216, cleanupEnabled=) at ../src/backend/access/transam/xlogarchive.c:177 [11:44:57.346] xlogpath = "pg_wal/RECOVERYXLOG", '\000' , "pg_wal/000000015", '0' , '\000' , "\b\000\000\000\000\000\000\000\204Wq(\027V\000\000\330~\276!\375\177\000\000xWq(\027V\000\000\064\357V(\027V\000\000\000\204\276!\375\177\000\000w\217q(\027V\000\000\330~\276!\375\177\000\000u\217q(\027V\000\000\030\231(\342\350\177\000\000\220\371V(\027V", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' ... [11:44:57.346] xlogRestoreCmd = 0x56172939e618 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/020_archive_status/data/t_020_archive_status_primary_data/archives/00000001", '0' , "5\" \"pg_wal/RECOVERYXLOG\"" [11:44:57.346] lastRestartPointFname = "00000001", '0' , "2\000\371V(\027V", '\000' , "\001\000\000y{\000\000\300\246\250\344\350\177\000\000`\247\250\344\350\177\000\000y{\000\000\002", '\000' , "\020\375\217\344\350\177\000\000\240w\276!\375\177", '\000' , "W\361\244\344\350\177\000\000O\361\244\344\350\177\000\000\030\346\071)\027V", '\000' , "@", '\000' ... [11:44:57.346] rc = [11:44:57.346] stat_buf = {st_dev = 94657461010712, st_ino = 335544320, st_nlink = 140638243728736, st_mode = 0, st_uid = 0, st_gid = 1003, __pad0 = 0, st_rdev = 406903504611511808, st_size = 16777216, st_blksize = 94657475896856, st_blocks = 140725169584544, st_atim = {tv_sec = 140725169584128, tv_nsec = 140725169584256}, st_mtim = {tv_sec = 0, tv_nsec = 94657475896856}, st_ctim = {tv_sec = 140638244455575, tv_nsec = 0}, __glibc_reserved = {94657461022516, 140725169587216, 94657462752977}} [11:44:57.346] restartSegNo = [11:44:57.346] restartRedoPtr = 33563696 [11:44:57.346] restartTli = 1 [11:44:57.346] __func__ = "RestoreArchivedFile" [11:44:57.346] #6 0x0000561727c24aef in XLogFileRead (segno=140725169587120, segno@entry=5, emode=emode@entry=13, tli=tli@entry=1, source=source@entry=XLOG_FROM_ARCHIVE, notfoundOk=notfoundOk@entry=true) at ../src/backend/access/transam/xlogrecovery.c:4220 [11:44:57.346] xlogfname = "00000001", '0' , "5\000history\000\000\000\t\000\000\000\000\200\006\214\344\350\177\000\000\300\207\276!\002\000\000\000\212V$\346\350\000\000" [11:44:57.346] activitymsg = "waiting for 00000001", '0' , "5\000\177\000\000\064\357V(\027V\000\000\260\340|(\027V\000\000\255\340|(\027V\000\000\030\207\276!\375\177\000\000\253\340|(\027V\000" [11:44:57.346] path = "P\203\276!\375\177\000\000P\203\276!\000\000\000\000\001\000\000\000\000\000\000\000\360\355V(\027V\000\000pg_wal/RECOVERYHISTORY\000\000\b", '\000' , "\b\364V(\027V\000\062", '\000' , "\b\000\000\000\000\000\000\000Y\374p(\027V\000\000\070\203\276!\375\177\000\000U\374p(\027V\000\000\064\357V(\027V\000\000a\374p(\027V\000\000Y\374p(\027V\000\000\070\203\276!\375\177\000\000U\374p(\027V\000\000\030\231(\342\350\177\000\000\322\370V(\027V", '\000' ... [11:44:57.346] fd = [11:44:57.346] __func__ = "XLogFileRead" [11:44:57.346] #7 0x0000561727c24cdf in XLogFileReadAnyTLI (segno=5, emode=emode@entry=13, source=XLOG_FROM_ANY) at ../src/backend/access/transam/xlogrecovery.c:4348 [11:44:57.346] hent = [11:44:57.346] tli = 1 [11:44:57.346] cell__state = {l = , i = } [11:44:57.346] path = "\367~q(\027V\000\000\030\231(\342\350\177\000\000@\207\276!\375\177\000\000t\001W(\027V\000\000\000\000\000\000\000\000\000\000\202\272\020)\027V\000\000\000\273\020)\027V\000\000\004\206\276!\375\177\000\000\330\337|(\027V\000\000\221\272\020)\027V\000\000\000\273\020)\027V\000\000$\206\276!\375\177\000\000\347\337|(\027V\000\000\000\273\020)\027V\000\000\002\000\000\000\000\000\000\000\061\tW(\027V\000\000&\206\276!\375\177\000\000$\206\276!\375\177", '\000' , "\177\177\177\030\000\000\000\060\000\000\000\000\206\276!\375\177\000\000\n\nW(\027V\000\000\000\000\000\000\000\000\000\000"... [11:44:57.346] cell = [11:44:57.346] fd = [11:44:57.346] tles = 0x5617293b0dc8 [11:44:57.346] __func__ = "XLogFileReadAnyTLI" [11:44:57.346] #8 0x0000561727c25edb in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=83886104, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=83886080, replayTLI=1, replayLSN=83886080, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3781 [11:44:57.346] oldSource = [11:44:57.346] startWalReceiver = false [11:44:57.346] last_fail_time = 765546053698534 [11:44:57.346] now = [11:44:57.346] streaming_reply_sent = false [11:44:57.346] __func__ = "WaitForWALToBecomeAvailable" [11:44:57.346] #9 0x0000561727c26a10 in XLogPageRead (xlogreader=0x56172934bf70, targetPagePtr=83886080, reqLen=24, targetRecPtr=83886080, readBuf=0x56172939ade8 "\025\321\006") at ../src/backend/access/transam/xlogrecovery.c:3362 [11:44:57.346] private = 0x56172934bf50 [11:44:57.346] emode = 15 [11:44:57.346] targetPageOff = 0 [11:44:57.346] targetSegNo = 5 [11:44:57.346] r = [11:44:57.346] __func__ = "XLogPageRead" [11:44:57.346] #10 0x0000561727c2efa4 in ReadPageInternal (state=state@entry=0x56172934bf70, pageptr=pageptr@entry=83886080, reqLen=reqLen@entry=24) at ../src/backend/access/transam/xlogreader.c:1060 [11:44:57.346] readLen = [11:44:57.346] targetPageOff = 0 [11:44:57.346] targetSegNo = 5 [11:44:57.346] hdr = [11:44:57.346] #11 0x0000561727c30bff in XLogDecodeNextRecord (state=state@entry=0x56172934bf70, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:597 [11:44:57.346] RecPtr = 83886080 [11:44:57.346] record = [11:44:57.346] targetPagePtr = 83886080 [11:44:57.346] randAccess = false [11:44:57.346] len = [11:44:57.346] total_len = [11:44:57.346] targetRecOff = 0 [11:44:57.346] pageHeaderSize = [11:44:57.346] assembled = false [11:44:57.346] gotheader = [11:44:57.346] readOff = [11:44:57.346] decoded = 0x0 [11:44:57.346] errormsg = 0x56172934bf70 "\253g\302'\027V" [11:44:57.346] #12 0x0000561727c317a1 in XLogReadAhead (state=0x56172934bf70, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:973 [11:44:57.346] result = [11:44:57.346] #13 0x0000561727c1bfaa in XLogPrefetcherNextBlock (pgsr_private=94657475562912, lsn=0x5617293b0798) at ../src/backend/access/transam/xlogprefetcher.c:492 [11:44:57.346] nonblocking = [11:44:57.346] record = [11:44:57.346] prefetcher = 0x56172934cda0 [11:44:57.346] reader = 0x56172934bf70 [11:44:57.346] replaying_lsn = 67109152 [11:44:57.346] __func__ = "XLogPrefetcherNextBlock" [11:44:57.346] #14 0x0000561727c20373 in lrq_prefetch (lrq=) at ../src/backend/access/transam/xlogprefetcher.c:252 [11:44:57.346] No locals. [11:44:57.346] #15 lrq_complete_lsn (lsn=83886080, lrq=0x5617293b0698) at ../src/backend/access/transam/xlogprefetcher.c:290 [11:44:57.346] No locals. [11:44:57.346] #16 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x56172934cda0, errmsg=errmsg@entry=0x7ffd21becb98) at ../src/backend/access/transam/xlogprefetcher.c:1037 [11:44:57.346] record = [11:44:57.346] replayed_up_to = 83886080 [11:44:57.346] #17 0x0000561727c2723d in ReadRecord (xlogprefetcher=0x56172934cda0, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=1) at ../src/backend/access/transam/xlogrecovery.c:3162 [11:44:57.346] errormsg = 0x56172811f7e6 "AUATUSH\203\354\bH\205\377\017\204:\001" [11:44:57.346] record = [11:44:57.346] xlogreader = 0x56172934bf70 [11:44:57.346] private = [11:44:57.366] __func__ = "ReadRecord" [11:44:57.366] #18 0x0000561727c2a461 in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1843 [11:44:57.366] xtime = [11:44:57.366] ru0 = {tv = {tv_sec = 1712230850, tv_usec = 136272}, ru = {ru_utime = {tv_sec = 0, tv_usec = 1343}, ru_stime = {tv_sec = 0, tv_usec = 0}, {ru_maxrss = 16004, __ru_maxrss_word = 16004}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 402, __ru_minflt_word = 402}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 2, __ru_nvcsw_word = 2}, {ru_nivcsw = 1, __ru_nivcsw_word = 1}}} [11:44:57.366] record = 0x7fe8e1b431f0 [11:44:57.366] reachedRecoveryTarget = false [11:44:57.366] replayTLI = 1 [11:44:57.366] __func__ = "PerformWalRecovery" [11:44:57.366] #19 0x0000561727c1403a in StartupXLOG () at ../src/backend/access/transam/xlog.c:5757 [11:44:57.366] Insert = [11:44:57.366] checkPoint = {redo = 33554472, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 740}, nextOid = 24576, nextMulti = 1, nextMultiOffset = 0, oldestXid = 731, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1712230848, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 740} [11:44:57.366] wasShutdown = false [11:44:57.366] didCrash = [11:44:57.366] haveTblspcMap = false [11:44:57.366] haveBackupLabel = false [11:44:57.366] EndOfLog = [11:44:57.366] EndOfLogTLI = [11:44:57.366] newTLI = [11:44:57.366] performedWalRecovery = [11:44:57.366] endOfRecoveryInfo = [11:44:57.366] abortedRecPtr = [11:44:57.366] missingContrecPtr = [11:44:57.366] oldestActiveXID = [11:44:57.366] promoted = false [11:44:57.366] __func__ = "StartupXLOG" [11:44:57.366] #20 0x00005617280d17c2 in StartupProcessMain (startup_data=, startup_data_len=) at ../src/backend/postmaster/startup.c:258 [11:44:57.366] No locals. [11:44:57.366] #21 0x00005617280c8dd9 in postmaster_child_launch (child_type=child_type@entry=B_STARTUP, startup_data=startup_data@entry=0x0, startup_data_len=startup_data_len@entry=0, client_sock=client_sock@entry=0x0) at ../src/backend/postmaster/launch_backend.c:265 [11:44:57.366] pid = [11:44:57.366] #22 0x00005617280cae57 in StartChildProcess (type=type@entry=B_STARTUP) at ../src/backend/postmaster/postmaster.c:3928 [11:44:57.366] pid = [11:44:57.366] __func__ = "StartChildProcess" [11:44:57.366] #23 0x00005617280d0f21 in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x56172934ae40) at ../src/backend/postmaster/postmaster.c:1364 [11:44:57.366] opt = [11:44:57.366] status = [11:44:57.366] userDoption = [11:44:57.366] listen_addr_saved = false [11:44:57.366] output_config_variable = [11:44:57.366] __func__ = "PostmasterMain" [11:44:57.366] #24 0x0000561727ef6b79 in main (argc=4, argv=0x56172934ae40) at ../src/backend/main/main.c:197 [11:44:57.366] do_check_root = [11:44:57.366] [11:44:57.366] Thread 1 (LWP 31776): [11:44:57.366] #0 0x00007fe8e48f2fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [11:44:57.366] No symbol table info available. [11:44:57.366] #1 0x00007fe8e49a413a in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [11:44:57.366] No symbol table info available. [11:44:57.366] #2 0x00007fe8e49b5a6f in clone () from /lib/x86_64-linux-gnu/libc.so.6 [11:44:57.366] No symbol table info available.