[00:56:30.654] src/tools/ci/cores_backtrace.sh linux /tmp/cores [00:56:31.371] dumping /tmp/cores/postgres-3-26154.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres [00:56:33.040] [New LWP 26154] [00:56:33.040] [New LWP 26081] [00:56:33.040] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [00:56:34.192] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [00:56:35.084] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [00:56:37.045] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [00:56:39.624] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [00:56:39.624] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo... [00:56:41.720] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [00:56:42.926] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [00:56:46.507] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [00:56:47.627] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [00:56:48.785] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [00:56:51.249] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [00:56:52.954] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [00:56:55.721] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [00:56:57.123] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [00:56:59.194] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [00:57:00.723] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [00:57:01.211] [Thread debugging using libthread_db enabled] [00:57:01.211] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [00:57:02.268] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [00:57:03.744] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [00:57:04.806] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [00:57:05.470] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [00:57:06.330] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [00:57:08.878] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [00:57:10.157] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [00:57:10.983] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [00:57:12.356] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [00:57:13.526] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [00:57:15.303] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [00:57:16.424] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [00:57:17.245] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [00:57:18.736] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [00:57:19.554] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [00:57:21.246] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [00:57:22.009] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [00:57:23.609] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [00:57:24.781] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [00:57:25.882] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [00:57:26.943] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [00:57:28.691] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [00:57:29.358] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [00:57:30.820] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [00:57:33.228] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [00:57:33.720] Core was generated by `postgres: standby2: startup waiting for 000000010000000000000003 '. [00:57:33.720] Program terminated with signal SIGQUIT, Quit. [00:57:33.720] #0 0x00007fc818e9ffc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [00:57:33.748] [Current thread is 1 (LWP 26154)] [00:57:33.870] [00:57:33.870] Thread 2 (Thread 0x7fc816836a40 (LWP 26081)): [00:57:33.870] #0 0x00007fc818f62a61 in clone () from /lib/x86_64-linux-gnu/libc.so.6 [00:57:33.870] No symbol table info available. [00:57:33.870] #1 0x00007fc818f50e5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [00:57:33.870] No symbol table info available. [00:57:33.870] #2 0x00007fc818f51497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [00:57:33.870] No symbol table info available. [00:57:33.870] #3 0x00007fc818f50cdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6 [00:57:33.870] No symbol table info available. [00:57:33.870] #4 0x00007fc818eaca69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [00:57:33.870] No symbol table info available. [00:57:33.870] #5 0x000055ba213d490b in RestoreArchivedFile (path=path@entry=0x7ffcaf6d3b50 "\360>m\257\374\177", xlogfname=xlogfname@entry=0x7ffcaf6d3fa0 "00000001", '0' , "3", recovername=recovername@entry=0x55ba21eab0dc "RECOVERYXLOG", expectedSize=16777216, cleanupEnabled=) at ../src/backend/access/transam/xlogarchive.c:178 [00:57:33.870] xlogpath = "pg_wal/RECOVERYXLOG", '\000' , "pg_wal/000000013", '0' , '\000' , "\b\000\000\000\000\000\000\000\304x\352!\272U\000\000x:m\257\374\177\000\000\270x\352!\272U\000\000\311\027\321!\272U\000\000\240?m\257\374\177\000\000\227\260\352!\272U\000\000x:m\257\374\177\000\000\225\260\352!\272U\000\000\030i\203\026\310\177\000\000%\"\321!\272U", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' ... [00:57:33.870] xlogRestoreCmd = 0x55ba23514838 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/020_archive_status/data/t_020_archive_status_primary_data/archives/00000001", '0' , "3\" \"pg_wal/RECOVERYXLOG\"" [00:57:33.870] lastRestartPointFname = "00000001", '0' , "2\000\"\321!\272U", '\000' , "\001\000\000\003f\000\000\300v\003\031\310\177\000\000`w\003\031\310\177\000\000\003f\000\000\002", '\000' , "\020\315\352\030\310\177\000\000@3m\257\374\177", '\000' , "W\301\377\030\310\177\000\000O\301\377\030\310\177\000\000\070HQ#\272U", '\000' , "@", '\000' ... [00:57:33.870] rc = [00:57:33.870] stat_buf = {st_dev = 94257919617453, st_ino = 335544320, st_nlink = 140497388174688, st_mode = 0, st_uid = 0, st_gid = 2002741104, __pad0 = 808414305, st_rdev = 1926442136728873984, st_size = 3472328296227680304, st_blksize = 94257944807480, st_blocks = 140723251656000, st_atim = {tv_sec = 140723251655584, tv_nsec = 140723251655712}, st_mtim = {tv_sec = 0, tv_nsec = 94257944807480}, st_ctim = {tv_sec = 140497388901527, tv_nsec = 0}, __glibc_reserved = {94257919629257, 140723251658672, 94257921292305}} [00:57:33.870] restartSegNo = [00:57:33.870] restartRedoPtr = 33563712 [00:57:33.870] restartTli = 1 [00:57:33.870] __func__ = "RestoreArchivedFile" [00:57:33.870] #6 0x000055ba213e359b in XLogFileRead (segno=140723251658576, segno@entry=3, emode=emode@entry=13, tli=tli@entry=1, source=source@entry=XLOG_FROM_ARCHIVE, notfoundOk=notfoundOk@entry=true) at ../src/backend/access/transam/xlogrecovery.c:4191 [00:57:33.870] xlogfname = "00000001", '0' , "3\000history\000\377\377\377", '\000' , "h\330\003\031\310\177\000\000 T\267\031\310\177\000" [00:57:33.870] activitymsg = "waiting for 00000001", '0' , "3\000\000\000\000\307\340\226!\272U\000\000\260?m\257\374\177\000\000\001\000\000\000\000\000\000\000\000\000\000\003\000\000\000\000\001\000\000\000\000\000\000" [00:57:33.870] path = "\360>m\257\374\177\000\000\360>m\257\000\000\000\000\001\000\000\000\000\000\000\000\205\026\321!\272U\000\000pg_wal/RECOVERYHISTORY\000\000\b", '\000' , "\235\034\321!\272U\000\062", '\000' , "\b\000\000\000\000\000\000\000I \352!\272U\000\000\330>m\257\374\177\000\000E \352!\272U\000\000\311\027\321!\272U\000\000Q \352!\272U\000\000I \352!\272U\000\000\330>m\257\374\177\000\000E \352!\272U\000\000\030i\203\026\310\177\000\000g!\321!\272U", '\000' ... [00:57:33.870] fd = [00:57:33.870] __func__ = "XLogFileRead" [00:57:33.870] #7 0x000055ba213e378b in XLogFileReadAnyTLI (segno=3, emode=emode@entry=13, source=XLOG_FROM_ANY) at ../src/backend/access/transam/xlogrecovery.c:4319 [00:57:33.870] hent = [00:57:33.870] tli = 1 [00:57:33.870] cell__state = {l = , i = } [00:57:33.870] path = "pg_wal/00000001", '0' , "3\000\230\000\000\000\000\000\000\000#\266~\032\310\177\000\000\001", '\000' , "h\330\003\031\310\177\000\000\000Dm\257\374\177\000\000\212&\177\032\310\177\000\000\310;\204\"\272U", '\000' , " \000\000\000\000\000\000\b\020Q#\272U\000\000\005\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000\060", '\000' , "\240\037\000\000\377\377\002", '\000' ... [00:57:33.870] cell = [00:57:33.870] fd = [00:57:33.870] tles = 0x55ba23526fe8 [00:57:33.870] __func__ = "XLogFileReadAnyTLI" [00:57:33.870] #8 0x000055ba213e4987 in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=50331672, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=50331648, replayTLI=1, replayLSN=50331648, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3752 [00:57:33.870] oldSource = [00:57:33.870] startWalReceiver = false [00:57:33.870] last_fail_time = 761619216797538 [00:57:33.870] now = [00:57:33.870] streaming_reply_sent = false [00:57:33.870] __func__ = "WaitForWALToBecomeAvailable" [00:57:33.870] #9 0x000055ba213e54bc in XLogPageRead (xlogreader=0x55ba234c1e70, targetPagePtr=50331648, reqLen=24, targetRecPtr=50331648, readBuf=0x55ba23511008 "\024\321\005") at ../src/backend/access/transam/xlogrecovery.c:3333 [00:57:33.870] private = 0x55ba234c1e50 [00:57:33.870] emode = 15 [00:57:33.870] targetPageOff = 0 [00:57:33.870] targetSegNo = 3 [00:57:33.870] r = [00:57:33.870] __func__ = "XLogPageRead" [00:57:33.870] #10 0x000055ba213ed8dc in ReadPageInternal (state=state@entry=0x55ba234c1e70, pageptr=pageptr@entry=50331648, reqLen=reqLen@entry=24) at ../src/backend/access/transam/xlogreader.c:1062 [00:57:33.870] readLen = [00:57:33.870] targetPageOff = 0 [00:57:33.870] targetSegNo = 3 [00:57:33.870] hdr = [00:57:33.870] #11 0x000055ba213ef537 in XLogDecodeNextRecord (state=state@entry=0x55ba234c1e70, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:599 [00:57:33.870] RecPtr = 50331648 [00:57:33.870] record = [00:57:33.870] targetPagePtr = 50331648 [00:57:33.870] randAccess = false [00:57:33.870] len = [00:57:33.870] total_len = [00:57:33.870] targetRecOff = 0 [00:57:33.870] pageHeaderSize = [00:57:33.870] assembled = false [00:57:33.870] gotheader = [00:57:33.870] readOff = [00:57:33.870] decoded = 0x0 [00:57:33.870] errormsg = 0x55ba213bfd4d "\204\300\017\205q\001" [00:57:33.870] #12 0x000055ba213f00d9 in XLogReadAhead (state=0x55ba234c1e70, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:975 [00:57:33.870] result = [00:57:33.870] #13 0x000055ba213daa56 in XLogPrefetcherNextBlock (pgsr_private=94257944472736, lsn=0x55ba235269b8) at ../src/backend/access/transam/xlogprefetcher.c:496 [00:57:33.870] nonblocking = [00:57:33.870] record = [00:57:33.870] prefetcher = 0x55ba234c2ca0 [00:57:33.870] reader = 0x55ba234c1e70 [00:57:33.870] replaying_lsn = 33563920 [00:57:33.870] __func__ = "XLogPrefetcherNextBlock" [00:57:33.870] #14 0x000055ba213dee1f in lrq_prefetch (lrq=) at ../src/backend/access/transam/xlogprefetcher.c:256 [00:57:33.870] No locals. [00:57:33.870] #15 lrq_complete_lsn (lsn=50331648, lrq=0x55ba235268b8) at ../src/backend/access/transam/xlogprefetcher.c:294 [00:57:33.870] No locals. [00:57:33.870] #16 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x55ba234c2ca0, errmsg=errmsg@entry=0x7ffcaf6d8738) at ../src/backend/access/transam/xlogprefetcher.c:1041 [00:57:33.870] record = [00:57:33.870] replayed_up_to = 50331648 [00:57:33.870] #17 0x000055ba213e5ce9 in ReadRecord (xlogprefetcher=0x55ba234c2ca0, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=1) at ../src/backend/access/transam/xlogrecovery.c:3133 [00:57:33.870] errormsg = 0x55ba218cfa83 "AUATUSH\203\354\bH\205\377\017\204:\001" [00:57:33.870] record = [00:57:33.870] xlogreader = 0x55ba234c1e70 [00:57:33.870] private = [00:57:33.870] __func__ = "ReadRecord" [00:57:33.870] #18 0x000055ba213e8e45 in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1814 [00:57:33.890] xtime = [00:57:33.890] ru0 = {tv = {tv_sec = 1708304016, tv_usec = 696033}, ru = {ru_utime = {tv_sec = 0, tv_usec = 0}, ru_stime = {tv_sec = 0, tv_usec = 1366}, {ru_maxrss = 15812, __ru_maxrss_word = 15812}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 399, __ru_minflt_word = 399}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 2, __ru_nvcsw_word = 2}, {ru_nivcsw = 2, __ru_nivcsw_word = 2}}} [00:57:33.890] record = 0x7fc81617c690 [00:57:33.890] reachedRecoveryTarget = false [00:57:33.890] replayTLI = 1 [00:57:33.890] __func__ = "PerformWalRecovery" [00:57:33.890] #19 0x000055ba213d2b94 in StartupXLOG () at ../src/backend/access/transam/xlog.c:5733 [00:57:33.890] Insert = [00:57:33.890] checkPoint = {redo = 33554472, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 739}, nextOid = 24576, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1708304014, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 739} [00:57:33.890] wasShutdown = false [00:57:33.890] didCrash = [00:57:33.890] haveTblspcMap = false [00:57:33.890] haveBackupLabel = false [00:57:33.890] EndOfLog = [00:57:33.890] EndOfLogTLI = [00:57:33.890] newTLI = [00:57:33.890] performedWalRecovery = [00:57:33.890] endOfRecoveryInfo = [00:57:33.890] abortedRecPtr = [00:57:33.890] missingContrecPtr = [00:57:33.890] oldestActiveXID = [00:57:33.890] promoted = false [00:57:33.890] __func__ = "StartupXLOG" [00:57:33.890] #20 0x000055ba21882e34 in StartupProcessMain () at ../src/backend/postmaster/startup.c:254 [00:57:33.890] No locals. [00:57:33.890] #21 0x000055ba2187329a in AuxiliaryProcessMain (auxtype=auxtype@entry=StartupProcess) at ../src/backend/postmaster/auxprocess.c:142 [00:57:33.890] __func__ = "AuxiliaryProcessMain" [00:57:33.890] #22 0x000055ba2187e180 in StartChildProcess (type=type@entry=StartupProcess) at ../src/backend/postmaster/postmaster.c:5311 [00:57:33.890] pid = [00:57:33.890] __func__ = "StartChildProcess" [00:57:33.890] #23 0x000055ba21882574 in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x55ba234c0d40) at ../src/backend/postmaster/postmaster.c:1459 [00:57:33.890] opt = [00:57:33.890] status = [00:57:33.890] userDoption = [00:57:33.890] listen_addr_saved = false [00:57:33.890] output_config_variable = [00:57:33.890] __func__ = "PostmasterMain" [00:57:33.890] #24 0x000055ba216ad84b in main (argc=4, argv=0x55ba234c0d40) at ../src/backend/main/main.c:198 [00:57:33.890] do_check_root = [00:57:33.890] [00:57:33.890] Thread 1 (LWP 26154): [00:57:33.890] #0 0x00007fc818e9ffc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [00:57:33.890] No symbol table info available. [00:57:33.890] #1 0x00007fc818f5113a in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [00:57:33.890] No symbol table info available. [00:57:33.890] #2 0x00007fc818f62a6f in clone () from /lib/x86_64-linux-gnu/libc.so.6 [00:57:33.890] No symbol table info available.