[22:18:51.363] src/tools/ci/cores_backtrace.sh linux /tmp/cores [22:18:52.090] dumping /tmp/cores/postgres-3-28984.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres [22:18:53.957] [New LWP 28984] [22:18:53.957] [New LWP 28831] [22:18:53.957] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [22:18:54.717] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [22:18:55.615] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [22:18:56.838] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [22:18:59.403] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [22:18:59.403] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo... [22:19:01.489] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [22:19:02.709] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [22:19:04.338] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [22:19:05.424] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [22:19:06.239] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [22:19:07.568] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [22:19:09.285] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [22:19:10.730] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [22:19:11.642] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [22:19:13.181] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [22:19:14.748] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [22:19:15.221] [Thread debugging using libthread_db enabled] [22:19:15.221] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [22:19:16.455] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [22:19:17.895] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [22:19:18.960] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [22:19:19.581] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [22:19:20.351] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [22:19:22.646] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [22:19:23.598] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [22:19:24.418] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [22:19:25.706] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [22:19:26.644] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [22:19:28.322] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [22:19:29.422] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [22:19:30.204] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [22:19:31.639] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [22:19:32.271] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [22:19:33.806] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [22:19:34.568] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [22:19:35.797] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [22:19:36.593] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [22:19:37.702] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [22:19:38.653] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [22:19:39.911] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [22:19:40.539] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [22:19:41.482] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [22:19:42.252] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [22:19:42.758] Core was generated by `postgres: replica_success: startup waiting for 000000010000000000000004 '. [22:19:42.758] Program terminated with signal SIGQUIT, Quit. [22:19:42.758] #0 0x00007f826be6ffc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [22:19:42.792] [Current thread is 1 (LWP 28984)] [22:19:42.922] [22:19:42.922] Thread 2 (Thread 0x7f8269806a40 (LWP 28831)): [22:19:42.922] #0 0x00007f826bf32a61 in clone () from /lib/x86_64-linux-gnu/libc.so.6 [22:19:42.922] No symbol table info available. [22:19:42.922] #1 0x00007f826bf20e5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [22:19:42.922] No symbol table info available. [22:19:42.922] #2 0x00007f826bf21497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [22:19:42.922] No symbol table info available. [22:19:42.922] #3 0x00007f826bf20cdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6 [22:19:42.922] No symbol table info available. [22:19:42.922] #4 0x00007f826be7ca69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [22:19:42.922] No symbol table info available. [22:19:42.922] #5 0x000055a4e9acd311 in RestoreArchivedFile (path=path@entry=0x7ffdf7738bd0 "p\217s\367\375\177", xlogfname=xlogfname@entry=0x7ffdf7739020 "00000001", '0' , "4", recovername=recovername@entry=0x55a4ea5dc9dc "RECOVERYXLOG", expectedSize=16777216, cleanupEnabled=) at ../src/backend/access/transam/xlogarchive.c:177 [22:19:42.922] xlogpath = "pg_wal/RECOVERYXLOG", '\000' , "pg_wal/000000014", '0' , '\000' , "\b\000\000\000\000\000\000\000̑]\352\244U\000\000\370\212s\367\375\177\000\000\300\221]\352\244U\000\000\267\037C\352\244U\000\000 \220s\367\375\177\000\000\227\311]\352\244U\000\000\370\212s\367\375\177\000\000\225\311]\352\244U\000\000\030i\200i\202\177\000\000\023*C\352\244U", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' ... [22:19:42.922] xlogRestoreCmd = 0x55a4eb5c1588 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/042_low_level_backup/data/t_042_low_level_backup_primary_data/archives/00000001", '0' , "4\" \"pg_wal/RECOVERYXLOG\"" [22:19:42.922] lastRestartPointFname = "00000001", '0' , "2\000*C\352\244U", '\000' , "\001\000\000\005q\000\000\300v\000l\202\177\000\000`w\000l\202\177\000\000\005q\000\000\002", '\000' , "\020\315\347k\202\177\000\000\300\203s\367\375\177", '\000' , "W\301\374k\202\177\000\000O\301\374k\202\177\000\000\210\025\\\353\244U", '\000' , "@", '\000' ... [22:19:42.922] rc = [22:19:42.922] stat_buf = {st_dev = 94166793253275, st_ino = 335544320, st_nlink = 140198132776288, st_mode = 0, st_uid = 0, st_gid = 1003, __pad0 = 0, st_rdev = 1456578917014263296, st_size = 16777216, st_blksize = 94166811678088, st_blocks = 140728754996672, st_atim = {tv_sec = 140728754996256, tv_nsec = 140728754996384}, st_mtim = {tv_sec = 0, tv_nsec = 94166811678088}, st_ctim = {tv_sec = 140198133503127, tv_nsec = 0}, __glibc_reserved = {94166793265079, 140728754999344, 94166794998041}} [22:19:42.922] restartSegNo = [22:19:42.922] restartRedoPtr = 33554472 [22:19:42.922] restartTli = 1 [22:19:42.922] __func__ = "RestoreArchivedFile" [22:19:42.922] #6 0x000055a4e9adbfa1 in XLogFileRead (segno=140728754999248, segno@entry=4, emode=emode@entry=13, tli=tli@entry=1, source=source@entry=XLOG_FROM_ARCHIVE, notfoundOk=notfoundOk@entry=true) at ../src/backend/access/transam/xlogrecovery.c:4209 [22:19:42.922] xlogfname = "00000001", '0' , "4\000&|m\202\177\000\000\b'\374\352\244U\000\000\244\001\000\000\000\000\000\000\001\000\000\000\000\000\000\000\250\345V\353\244U\000" [22:19:42.922] activitymsg = "waiting for 00000001", '0' , "4\000\000\000\000\244\001\000\000\000\000\000\000\001\000\000\000\000\000\000\000#\266{m\202\177\000\000\001", '\000' [22:19:42.922] path = "p\217s\367\375\177\000\000p\217s\367\000\000\000\000\001\000\000\000\000\000\000\000s\036C\352\244U\000\000pg_wal/RECOVERYHISTORY\000\000\b", '\000' , "\213$C\352\244U\000\062", '\000' , "\b\000\000\000\000\000\000\000\271\067]\352\244U\000\000X\217s\367\375\177\000\000\265\067]\352\244U\000\000\267\037C\352\244U\000\000\301\067]\352\244U\000\000\271\067]\352\244U\000\000X\217s\367\375\177\000\000\265\067]\352\244U\000\000\030i\200i\202\177\000\000U)C\352\244U", '\000' ... [22:19:42.922] fd = [22:19:42.922] __func__ = "XLogFileRead" [22:19:42.922] #7 0x000055a4e9adc191 in XLogFileReadAnyTLI (segno=4, emode=emode@entry=13, source=XLOG_FROM_ANY) at ../src/backend/access/transam/xlogrecovery.c:4337 [22:19:42.922] hent = [22:19:42.922] tli = 1 [22:19:42.922] cell__state = {l = , i = } [22:19:42.922] path = "\001\000\000\000\000\000\000\000\004\000\000\t", '\000' , "\367\061C\352\244U\000\000\000\000\000\000\000\000\000\000\002\227\375\352\244U\000\000\200\227\375\352\244U\000\000\240\037\000\000\377\377\002\000\030\"i\352\244U\000\000\021\227\375\352\244U\000\000\200\227\375\352\244U\000\000D\222s\367\375\177\000\000'\"i\352\244U\000\000\200\227\375\352\244U\000\000\002\000\000\000\000\000\000\000\264\071C\352\244U\000\000", '\177' , "\000\000\000\000\000\000\000\000\000\000\377\377", '\000' , "\377\000\000\000\000\000LC_MESSAGES\000LC_P", '\\' ... [22:19:42.922] cell = [22:19:42.922] fd = [22:19:42.922] tles = 0x55a4eb5d3d38 [22:19:42.922] __func__ = "XLogFileReadAnyTLI" [22:19:42.922] #8 0x000055a4e9add38d in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=67108888, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=67108864, replayTLI=1, replayLSN=67108864, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3770 [22:19:42.922] oldSource = [22:19:42.922] startWalReceiver = false [22:19:42.922] last_fail_time = 767139369051795 [22:19:42.922] now = [22:19:42.922] streaming_reply_sent = false [22:19:42.922] __func__ = "WaitForWALToBecomeAvailable" [22:19:42.922] #9 0x000055a4e9addec2 in XLogPageRead (xlogreader=0x55a4eb56eed0, targetPagePtr=67108864, reqLen=24, targetRecPtr=67108864, readBuf=0x55a4eb5bdd58 "\025\321\002") at ../src/backend/access/transam/xlogrecovery.c:3351 [22:19:42.922] private = 0x55a4eb56eeb0 [22:19:42.922] emode = 15 [22:19:42.922] targetPageOff = 0 [22:19:42.922] targetSegNo = 4 [22:19:42.922] r = [22:19:42.922] __func__ = "XLogPageRead" [22:19:42.922] #10 0x000055a4e9ae631d in ReadPageInternal (state=state@entry=0x55a4eb56eed0, pageptr=pageptr@entry=67108864, reqLen=reqLen@entry=24) at ../src/backend/access/transam/xlogreader.c:1060 [22:19:42.922] readLen = [22:19:42.922] targetPageOff = 0 [22:19:42.922] targetSegNo = 4 [22:19:42.922] hdr = [22:19:42.922] #11 0x000055a4e9ae7f78 in XLogDecodeNextRecord (state=state@entry=0x55a4eb56eed0, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:597 [22:19:42.922] RecPtr = 67108864 [22:19:42.922] record = [22:19:42.922] targetPagePtr = 67108864 [22:19:42.922] randAccess = false [22:19:42.922] len = [22:19:42.922] total_len = [22:19:42.922] targetRecOff = 0 [22:19:42.922] pageHeaderSize = [22:19:42.922] assembled = false [22:19:42.922] gotheader = [22:19:42.922] readOff = [22:19:42.922] decoded = 0x0 [22:19:42.922] errormsg = 0x40000000062 [22:19:42.922] #12 0x000055a4e9ae8b1a in XLogReadAhead (state=0x55a4eb56eed0, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:973 [22:19:42.922] result = [22:19:42.922] #13 0x000055a4e9ad345c in XLogPrefetcherNextBlock (pgsr_private=94166811344128, lsn=0x55a4eb5d38b8) at ../src/backend/access/transam/xlogprefetcher.c:492 [22:19:42.922] nonblocking = [22:19:42.922] record = [22:19:42.922] prefetcher = 0x55a4eb56fd00 [22:19:42.922] reader = 0x55a4eb56eed0 [22:19:42.922] replaying_lsn = 50331936 [22:19:42.922] __func__ = "XLogPrefetcherNextBlock" [22:19:42.922] #14 0x000055a4e9ad7825 in lrq_prefetch (lrq=) at ../src/backend/access/transam/xlogprefetcher.c:252 [22:19:42.922] No locals. [22:19:42.922] #15 lrq_complete_lsn (lsn=67108864, lrq=0x55a4eb5d3608) at ../src/backend/access/transam/xlogprefetcher.c:290 [22:19:42.922] No locals. [22:19:42.922] #16 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x55a4eb56fd00, errmsg=errmsg@entry=0x7ffdf773d7b8) at ../src/backend/access/transam/xlogprefetcher.c:1037 [22:19:42.922] record = [22:19:42.922] replayed_up_to = 67108864 [22:19:42.922] #17 0x000055a4e9ade6ef in ReadRecord (xlogprefetcher=0x55a4eb56fd00, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=1) at ../src/backend/access/transam/xlogrecovery.c:3151 [22:19:42.922] errormsg = 0x55a4e9fdf015 "AUATUSH\203\354\bH\205\377\017\204:\001" [22:19:42.941] record = [22:19:42.941] xlogreader = 0x55a4eb56eed0 [22:19:42.941] private = [22:19:42.941] __func__ = "ReadRecord" [22:19:42.941] #18 0x000055a4e9ae1886 in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1832 [22:19:42.941] xtime = [22:19:42.941] ru0 = {tv = {tv_sec = 1713824168, tv_usec = 289819}, ru = {ru_utime = {tv_sec = 0, tv_usec = 1738}, ru_stime = {tv_sec = 0, tv_usec = 0}, {ru_maxrss = 16476, __ru_maxrss_word = 16476}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 407, __ru_minflt_word = 407}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 5, __ru_nvcsw_word = 5}, {ru_nivcsw = 1, __ru_nivcsw_word = 1}}} [22:19:42.941] record = 0x7f82690dae38 [22:19:42.941] reachedRecoveryTarget = false [22:19:42.941] replayTLI = 1 [22:19:42.941] __func__ = "PerformWalRecovery" [22:19:42.941] #19 0x000055a4e9acb2a9 in StartupXLOG () at ../src/backend/access/transam/xlog.c:5806 [22:19:42.941] Insert = [22:19:42.941] checkPoint = {redo = 33554472, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 738}, nextOid = 14202, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1713824164, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 738} [22:19:42.941] wasShutdown = false [22:19:42.941] didCrash = [22:19:42.941] haveTblspcMap = false [22:19:42.941] haveBackupLabel = true [22:19:42.941] EndOfLog = [22:19:42.941] EndOfLogTLI = [22:19:42.941] newTLI = [22:19:42.941] performedWalRecovery = [22:19:42.941] endOfRecoveryInfo = [22:19:42.941] abortedRecPtr = [22:19:42.941] missingContrecPtr = [22:19:42.941] oldestActiveXID = [22:19:42.941] promoted = false [22:19:42.941] __func__ = "StartupXLOG" [22:19:42.941] #20 0x000055a4e9f910be in StartupProcessMain (startup_data=, startup_data_len=) at ../src/backend/postmaster/startup.c:258 [22:19:42.941] No locals. [22:19:42.941] #21 0x000055a4e9f886da in postmaster_child_launch (child_type=child_type@entry=B_STARTUP, startup_data=startup_data@entry=0x0, startup_data_len=startup_data_len@entry=0, client_sock=client_sock@entry=0x0) at ../src/backend/postmaster/launch_backend.c:265 [22:19:42.941] pid = [22:19:42.941] #22 0x000055a4e9f8a753 in StartChildProcess (type=type@entry=B_STARTUP) at ../src/backend/postmaster/postmaster.c:3928 [22:19:42.941] pid = [22:19:42.941] __func__ = "StartChildProcess" [22:19:42.941] #23 0x000055a4e9f9081d in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x55a4eb56dd60) at ../src/backend/postmaster/postmaster.c:1364 [22:19:42.941] opt = [22:19:42.941] status = [22:19:42.941] userDoption = [22:19:42.941] listen_addr_saved = false [22:19:42.941] output_config_variable = [22:19:42.941] __func__ = "PostmasterMain" [22:19:42.941] #24 0x000055a4e9db11cd in main (argc=4, argv=0x55a4eb56dd60) at ../src/backend/main/main.c:197 [22:19:42.941] do_check_root = [22:19:42.941] [22:19:42.941] Thread 1 (LWP 28984): [22:19:42.941] #0 0x00007f826be6ffc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [22:19:42.941] No symbol table info available. [22:19:42.941] #1 0x00007f826bf2113a in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [22:19:42.941] No symbol table info available. [22:19:42.941] #2 0x00007f826bf32a6f in clone () from /lib/x86_64-linux-gnu/libc.so.6 [22:19:42.941] No symbol table info available.