[04:45:29.411] src/tools/ci/cores_backtrace.sh linux /tmp/cores [04:45:30.123] dumping /tmp/cores/postgres-3-27357.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres [04:45:31.967] [New LWP 27357] [04:45:31.967] [New LWP 26335] [04:45:31.967] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [04:45:32.731] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [04:45:33.655] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [04:45:34.919] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [04:45:37.536] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [04:45:37.536] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo... [04:45:39.650] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [04:45:40.860] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [04:45:42.513] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [04:45:43.611] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [04:45:44.410] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [04:45:45.736] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [04:45:47.453] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [04:45:48.929] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [04:45:49.846] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [04:45:51.394] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [04:45:52.972] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [04:45:53.450] [Thread debugging using libthread_db enabled] [04:45:53.450] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [04:45:54.700] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [04:45:56.154] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [04:45:57.240] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [04:45:57.895] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [04:45:58.771] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [04:46:01.310] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [04:46:02.222] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [04:46:03.014] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [04:46:04.291] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [04:46:05.211] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [04:46:06.881] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [04:46:07.975] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [04:46:08.767] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [04:46:10.202] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [04:46:10.842] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [04:46:12.375] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [04:46:13.139] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [04:46:14.366] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [04:46:15.159] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [04:46:16.265] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [04:46:17.226] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [04:46:18.461] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [04:46:19.082] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [04:46:20.017] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [04:46:20.811] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [04:46:21.306] Core was generated by `postgres: standby: startup waiting for 000000010000000000000005 '. [04:46:21.306] Program terminated with signal SIGQUIT, Quit. [04:46:21.306] #0 0x00007f38566c8fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [04:46:21.335] [Current thread is 1 (LWP 27357)] [04:46:21.458] [04:46:21.458] Thread 2 (Thread 0x7f385405fa40 (LWP 26335)): [04:46:21.458] #0 0x00007f385678ba61 in clone () from /lib/x86_64-linux-gnu/libc.so.6 [04:46:21.458] No symbol table info available. [04:46:21.458] #1 0x00007f3856779e5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [04:46:21.458] No symbol table info available. [04:46:21.458] #2 0x00007f385677a497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [04:46:21.458] No symbol table info available. [04:46:21.458] #3 0x00007f3856779cdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6 [04:46:21.458] No symbol table info available. [04:46:21.458] #4 0x00007f38566d5a69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [04:46:21.458] No symbol table info available. [04:46:21.458] #5 0x00005583a4ac0275 in RestoreArchivedFile (path=path@entry=0x7ffe5bb88500 "\240\210\270[\376\177", xlogfname=xlogfname@entry=0x7ffe5bb88950 "00000001", '0' , "5", recovername=recovername@entry=0x5583a55d3adc "RECOVERYXLOG", expectedSize=16777216, cleanupEnabled=) at ../src/backend/access/transam/xlogarchive.c:177 [04:46:21.458] xlogpath = "pg_wal/RECOVERYXLOG", '\000' , "pg_wal/000000015", '0' , '\000' , "\b\000\000\000\000\000\000\000\314\002]\245\203U\000\000(\204\270[\376\177\000\000\300\002]\245\203U\000\000\064\215B\245\203U\000\000P\211\270[\376\177\000\000\227:]\245\203U\000\000(\204\270[\376\177\000\000\225:]\245\203U\000\000\030\371\005T8\177\000\000\220\227B\245\203U", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' ... [04:46:21.458] xlogRestoreCmd = 0x5583a73f9538 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/020_archive_status/data/t_020_archive_status_primary_data/archives/00000001", '0' , "5\" \"pg_wal/RECOVERYXLOG\"" [04:46:21.458] lastRestartPointFname = "00000001", '0' , "2\000\227B\245\203U", '\000' , "\001\000\000\207j\000\000\300\006\206V8\177\000\000`\a\206V8\177\000\000\207j\000\000\002", '\000' , "\020]mV8\177\000\000\360|\270[\376\177", '\000' , "WQ\202V8\177\000\000OQ\202V8\177\000\000\070\225?\247\203U", '\000' , "@", '\000' ... [04:46:21.458] rc = [04:46:21.458] stat_buf = {st_dev = 94023901667096, st_ino = 335544320, st_nlink = 139879944850784, st_mode = 0, st_uid = 0, st_gid = 1003, __pad0 = 0, st_rdev = 15672067676381047296, st_size = 16777216, st_blksize = 94023935038776, st_blocks = 140730437238512, st_atim = {tv_sec = 140730437238096, tv_nsec = 140730437238224}, st_mtim = {tv_sec = 0, tv_nsec = 94023935038776}, st_ctim = {tv_sec = 139879945577623, tv_nsec = 0}, __glibc_reserved = {94023901678900, 140730437241184, 94023903412761}} [04:46:21.458] restartSegNo = [04:46:21.458] restartRedoPtr = 33563696 [04:46:21.458] restartTli = 1 [04:46:21.458] __func__ = "RestoreArchivedFile" [04:46:21.458] #6 0x00005583a4acef05 in XLogFileRead (segno=140730437241088, segno@entry=5, emode=emode@entry=13, tli=tli@entry=1, source=source@entry=XLOG_FROM_ARCHIVE, notfoundOk=notfoundOk@entry=true) at ../src/backend/access/transam/xlogrecovery.c:4209 [04:46:21.458] xlogfname = "00000001", '0' , "5\000history\000\215\270[\376\177\000\000\212\266\001X8\177\000\000\350\302\373\245\002\000\000\000\357\001\000\000\000\000\000" [04:46:21.458] activitymsg = "waiting for 00000001", '0' , "5\000\177\000\000\064\215B\245\203U\000\000\020\242h\245\203U\000\000\r\242h\245\203U\000\000h\214\270[\376\177\000\000\v\242h\245\203U\000" [04:46:21.458] path = "\240\210\270[\376\177\000\000\240\210\270[\000\000\000\000\001\000\000\000\000\000\000\000\360\213B\245\203U\000\000pg_wal/RECOVERYHISTORY\000\000\b", '\000' , "\b\222B\245\203U\000\062", '\000' , "\b\000\000\000\000\000\000\000\071\247\\\245\203U\000\000\210\210\270[\376\177\000\000\065\247\\\245\203U\000\000\064\215B\245\203U\000\000A\247\\\245\203U\000\000\071\247\\\245\203U\000\000\210\210\270[\376\177\000\000\065\247\\\245\203U\000\000\030\371\005T8\177\000\000ҖB\245\203U", '\000' ... [04:46:21.458] fd = [04:46:21.458] __func__ = "XLogFileRead" [04:46:21.458] #7 0x00005583a4acf0f5 in XLogFileReadAnyTLI (segno=5, emode=emode@entry=13, source=XLOG_FROM_ANY) at ../src/backend/access/transam/xlogrecovery.c:4337 [04:46:21.458] hent = [04:46:21.458] tli = 1 [04:46:21.458] cell__state = {l = , i = } [04:46:21.458] path = "\027*]\245\203U\000\000\030\371\005T8\177\000\000\220\214\270[\376\177\000\000t\237B\245\203U\000\000\000\000\000\000\000\000\000\000\002\063\375\245\203U\000\000\200\063\375\245\203U\000\000T\213\270[\376\177\000\000\070\241h\245\203U\000\000\021\063\375\245\203U\000\000\200\063\375\245\203U\000\000t\213\270[\376\177\000\000G\241h\245\203U\000\000\200\063\375\245\203U\000\000\002\000\000\000\000\000\000\000\061\247B\245\203U\000\000v\213\270[\376\177\000\000t\213\270[\376\177", '\000' , "\030\000\000\000\060\000\000\000P\213\270[\376\177\000\000\n\250B\245\203U\000\000LC_MESSA"... [04:46:21.458] cell = [04:46:21.458] fd = [04:46:21.458] tles = 0x5583a740bce8 [04:46:21.458] __func__ = "XLogFileReadAnyTLI" [04:46:21.458] #8 0x00005583a4ad02f1 in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=83886104, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=83886080, replayTLI=1, replayLSN=83886080, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3770 [04:46:21.458] oldSource = [04:46:21.458] startWalReceiver = false [04:46:21.458] last_fail_time = 766471344778344 [04:46:21.458] now = [04:46:21.458] streaming_reply_sent = false [04:46:21.458] __func__ = "WaitForWALToBecomeAvailable" [04:46:21.458] #9 0x00005583a4ad0e26 in XLogPageRead (xlogreader=0x5583a73a6eb0, targetPagePtr=83886080, reqLen=24, targetRecPtr=83886080, readBuf=0x5583a73f5d08 "\025\321\002") at ../src/backend/access/transam/xlogrecovery.c:3351 [04:46:21.458] private = 0x5583a73a6e90 [04:46:21.458] emode = 15 [04:46:21.458] targetPageOff = 0 [04:46:21.458] targetSegNo = 5 [04:46:21.458] r = [04:46:21.458] __func__ = "XLogPageRead" [04:46:21.458] #10 0x00005583a4ad9281 in ReadPageInternal (state=state@entry=0x5583a73a6eb0, pageptr=pageptr@entry=83886080, reqLen=reqLen@entry=24) at ../src/backend/access/transam/xlogreader.c:1060 [04:46:21.458] readLen = [04:46:21.458] targetPageOff = 0 [04:46:21.458] targetSegNo = 5 [04:46:21.458] hdr = [04:46:21.458] #11 0x00005583a4adaedc in XLogDecodeNextRecord (state=state@entry=0x5583a73a6eb0, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:597 [04:46:21.458] RecPtr = 83886080 [04:46:21.458] record = [04:46:21.458] targetPagePtr = 83886080 [04:46:21.458] randAccess = false [04:46:21.458] len = [04:46:21.458] total_len = [04:46:21.458] targetRecOff = 0 [04:46:21.458] pageHeaderSize = [04:46:21.458] assembled = false [04:46:21.458] gotheader = [04:46:21.458] readOff = [04:46:21.458] decoded = 0x0 [04:46:21.458] errormsg = 0x5583a73a6eb0 "\301\v\255\244\203U" [04:46:21.458] #12 0x00005583a4adba7e in XLogReadAhead (state=0x5583a73a6eb0, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:973 [04:46:21.458] result = [04:46:21.458] #13 0x00005583a4ac63c0 in XLogPrefetcherNextBlock (pgsr_private=94023934704864, lsn=0x5583a740b6b8) at ../src/backend/access/transam/xlogprefetcher.c:492 [04:46:21.458] nonblocking = [04:46:21.458] record = [04:46:21.458] prefetcher = 0x5583a73a7ce0 [04:46:21.458] reader = 0x5583a73a6eb0 [04:46:21.458] replaying_lsn = 67109152 [04:46:21.458] __func__ = "XLogPrefetcherNextBlock" [04:46:21.458] #14 0x00005583a4aca789 in lrq_prefetch (lrq=) at ../src/backend/access/transam/xlogprefetcher.c:252 [04:46:21.458] No locals. [04:46:21.458] #15 lrq_complete_lsn (lsn=83886080, lrq=0x5583a740b5b8) at ../src/backend/access/transam/xlogprefetcher.c:290 [04:46:21.458] No locals. [04:46:21.458] #16 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x5583a73a7ce0, errmsg=errmsg@entry=0x7ffe5bb8d0e8) at ../src/backend/access/transam/xlogprefetcher.c:1037 [04:46:21.458] record = [04:46:21.458] replayed_up_to = 83886080 [04:46:21.458] #17 0x00005583a4ad1653 in ReadRecord (xlogprefetcher=0x5583a73a7ce0, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=1) at ../src/backend/access/transam/xlogrecovery.c:3151 [04:46:21.458] errormsg = 0x5583a4fd0fc4 "AUATUSH\203\354\bH\205\377\017\204:\001" [04:46:21.458] record = [04:46:21.458] xlogreader = 0x5583a73a6eb0 [04:46:21.479] private = [04:46:21.479] __func__ = "ReadRecord" [04:46:21.479] #18 0x00005583a4ad47ea in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1832 [04:46:21.479] xtime = [04:46:21.479] ru0 = {tv = {tv_sec = 1713156141, tv_usec = 226266}, ru = {ru_utime = {tv_sec = 0, tv_usec = 1413}, ru_stime = {tv_sec = 0, tv_usec = 0}, {ru_maxrss = 16060, __ru_maxrss_word = 16060}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 402, __ru_minflt_word = 402}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 2, __ru_nvcsw_word = 2}, {ru_nivcsw = 1, __ru_nivcsw_word = 1}}} [04:46:21.479] record = 0x7f38539191f0 [04:46:21.479] reachedRecoveryTarget = false [04:46:21.479] replayTLI = 1 [04:46:21.479] __func__ = "PerformWalRecovery" [04:46:21.479] #19 0x00005583a4abe20d in StartupXLOG () at ../src/backend/access/transam/xlog.c:5806 [04:46:21.479] Insert = [04:46:21.479] checkPoint = {redo = 33554472, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 739}, nextOid = 24576, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1713156139, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 739} [04:46:21.479] wasShutdown = false [04:46:21.479] didCrash = [04:46:21.479] haveTblspcMap = false [04:46:21.479] haveBackupLabel = false [04:46:21.479] EndOfLog = [04:46:21.479] EndOfLogTLI = [04:46:21.479] newTLI = [04:46:21.479] performedWalRecovery = [04:46:21.479] endOfRecoveryInfo = [04:46:21.479] abortedRecPtr = [04:46:21.479] missingContrecPtr = [04:46:21.479] oldestActiveXID = [04:46:21.479] promoted = false [04:46:21.479] __func__ = "StartupXLOG" [04:46:21.479] #20 0x00005583a4f8306d in StartupProcessMain (startup_data=, startup_data_len=) at ../src/backend/postmaster/startup.c:258 [04:46:21.479] No locals. [04:46:21.479] #21 0x00005583a4f7a689 in postmaster_child_launch (child_type=child_type@entry=B_STARTUP, startup_data=startup_data@entry=0x0, startup_data_len=startup_data_len@entry=0, client_sock=client_sock@entry=0x0) at ../src/backend/postmaster/launch_backend.c:265 [04:46:21.479] pid = [04:46:21.479] #22 0x00005583a4f7c702 in StartChildProcess (type=type@entry=B_STARTUP) at ../src/backend/postmaster/postmaster.c:3928 [04:46:21.479] pid = [04:46:21.479] __func__ = "StartChildProcess" [04:46:21.479] #23 0x00005583a4f827cc in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x5583a73a5d80) at ../src/backend/postmaster/postmaster.c:1364 [04:46:21.479] opt = [04:46:21.479] status = [04:46:21.479] userDoption = [04:46:21.479] listen_addr_saved = false [04:46:21.479] output_config_variable = [04:46:21.479] __func__ = "PostmasterMain" [04:46:21.479] #24 0x00005583a4da33cd in main (argc=4, argv=0x5583a73a5d80) at ../src/backend/main/main.c:197 [04:46:21.479] do_check_root = [04:46:21.479] [04:46:21.479] Thread 1 (LWP 27357): [04:46:21.479] #0 0x00007f38566c8fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [04:46:21.479] No symbol table info available. [04:46:21.479] #1 0x00007f385677a13a in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [04:46:21.479] No symbol table info available. [04:46:21.479] #2 0x00007f385678ba6f in clone () from /lib/x86_64-linux-gnu/libc.so.6 [04:46:21.479] No symbol table info available.