[18:12:04.729] src/tools/ci/cores_backtrace.sh linux /tmp/cores [18:12:05.438] dumping /tmp/cores/postgres-3-36172.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres [18:12:08.552] [New LWP 36172] [18:12:08.552] [New LWP 35103] [18:12:08.552] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [18:12:09.324] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [18:12:10.253] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [18:12:11.537] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [18:12:14.152] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [18:12:14.152] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo... [18:12:16.260] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [18:12:17.473] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [18:12:19.137] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [18:12:20.238] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [18:12:21.031] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [18:12:22.382] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [18:12:24.093] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [18:12:25.573] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [18:12:26.483] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [18:12:28.013] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [18:12:29.552] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [18:12:30.022] [Thread debugging using libthread_db enabled] [18:12:30.022] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [18:12:31.231] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [18:12:32.659] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [18:12:33.720] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [18:12:34.354] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [18:12:35.231] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [18:12:37.716] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [18:12:38.666] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [18:12:39.486] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [18:12:40.751] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [18:12:41.700] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [18:12:43.395] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [18:12:44.514] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [18:12:45.306] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [18:12:46.759] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [18:12:47.396] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [18:12:48.923] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [18:12:49.684] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [18:12:50.906] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [18:12:51.720] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [18:12:52.835] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [18:12:53.805] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [18:12:55.076] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [18:12:55.715] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [18:12:56.673] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [18:12:57.472] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [18:12:57.981] Core was generated by `postgres: standby2: startup waiting for 000000010000000000000006 '. [18:12:57.981] Program terminated with signal SIGQUIT, Quit. [18:12:57.981] #0 0x00007f7dc4f06fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [18:12:58.009] [Current thread is 1 (LWP 36172)] [18:12:58.124] [18:12:58.124] Thread 2 (Thread 0x7f7dc289da40 (LWP 35103)): [18:12:58.124] #0 0x00007f7dc4fc9a61 in clone () from /lib/x86_64-linux-gnu/libc.so.6 [18:12:58.124] No symbol table info available. [18:12:58.124] #1 0x00007f7dc4fb7e5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [18:12:58.124] No symbol table info available. [18:12:58.124] #2 0x00007f7dc4fb8497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [18:12:58.124] No symbol table info available. [18:12:58.124] #3 0x00007f7dc4fb7cdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6 [18:12:58.124] No symbol table info available. [18:12:58.124] #4 0x00007f7dc4f13a69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [18:12:58.124] No symbol table info available. [18:12:58.124] #5 0x000056416a495be0 in RestoreArchivedFile (path=path@entry=0x7ffecab9fe70 "pg_wal/00000", xlogfname=xlogfname@entry=0x7ffecaba02c0 "00000001", '0' , "6", recovername=recovername@entry=0x56416af695dc "RECOVERYXLOG", expectedSize=16777216, cleanupEnabled=) at ../src/backend/access/transam/xlogarchive.c:177 [18:12:58.124] xlogpath = "pg_wal/RECOVERYXLOG", '\000' , "\066", '\000' , "\b\000\000\000\000\000\000\000\274]\366jAV\000\000\230\375\271\312\376\177\000\000\260]\366jAV\000\000\355\004\335jAV\000\000\300\002\272\312\376\177\000\000\227\225\366jAV\000\000\230\375\271\312\376\177\000\000\225\225\366jAV\000\000\030ى\302}\177\000\000I\017\335jAV", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' ... [18:12:58.124] xlogRestoreCmd = 0x56416d9c4638 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/040_standby_failover_slots_sync/data/t_040_standby_failover_slots_sync_publisher_data/archives/00000001", '0' , "6\" \"pg_wal/RECOVERYXLOG\"" [18:12:58.124] lastRestartPointFname = "00000001", '0' , "5\000\017\335jAV", '\000' , "\001\000\000\373\214\000\000\300\346\t\305}\177\000\000`\347\t\305}\177\000\000\373\214\000\000\002", '\000' , "\020=\361\304}\177\000\000`\366\271\312\376\177", '\000' , "W1\006\305}\177\000\000O1\006\305}\177\000\000\070F\234mAV", '\000' , "@", '\000' ... [18:12:58.124] rc = [18:12:58.124] stat_buf = {st_dev = 94838965720785, st_ino = 335544320, st_nlink = 140178151730528, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 9985148211913828096, st_size = 0, st_blksize = 94839011821112, st_blocks = 140732299606112, st_atim = {tv_sec = 140732299605696, tv_nsec = 140732299605824}, st_mtim = {tv_sec = 0, tv_nsec = 94839011821112}, st_ctim = {tv_sec = 140178152457367, tv_nsec = 0}, __glibc_reserved = {94838965732589, 140732299608784, 94838967393545}} [18:12:58.124] restartSegNo = [18:12:58.124] restartRedoPtr = 83886120 [18:12:58.124] restartTli = 1 [18:12:58.124] __func__ = "RestoreArchivedFile" [18:12:58.124] #6 0x000056416a4a4870 in XLogFileRead (segno=140732299608688, segno@entry=6, emode=emode@entry=13, tli=tli@entry=1, source=source@entry=XLOG_FROM_ARCHIVE, notfoundOk=notfoundOk@entry=true) at ../src/backend/access/transam/xlogrecovery.c:4209 [18:12:58.124] xlogfname = "00000001", '0' , "6\000s&kAV\000\000\004\000\000\t\000\000\000\000\004\000\000\t", '\000' , "\305\001\000\000\000\000\000" [18:12:58.124] activitymsg = "waiting for 00000001", '0' , "6\000\177\000\000 ɽ\305}\177\000\000\060\301\001kAV\000\000-\301\001kAV\000\000\330\005\272\312\376\177\000\000+\301\001kAV\000" [18:12:58.124] path = "pg_wal/00000\000\000\000\000", '\060' , "\066\000pg_wal/RECOVERYHISTORY\000\000\b\000\000\000\000\000\000\066\000\000\000\000\000\000\000\000\301\t\335j\000\000\000\062\000\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000\274]\366jAV\000\000\b\000\000\000\000\000\000\000\071\003\366jAV\000\000\370\001\272\312\376\177\000\000\065\003\366jAV\000\000\355\004\335jAV\000\000A\003\366jAV\000\000\071\003\366jAV\000\000\370\001\272\312\376\177\000\000\065\003\366jAV\000\000\030ى\302}\177\000\000\213\016\335jAV", '\000' ... [18:12:58.124] fd = [18:12:58.124] __func__ = "XLogFileRead" [18:12:58.124] #7 0x000056416a4a4a60 in XLogFileReadAnyTLI (segno=6, emode=emode@entry=13, source=XLOG_FROM_ANY) at ../src/backend/access/transam/xlogrecovery.c:4337 [18:12:58.124] hent = [18:12:58.124] tli = 1 [18:12:58.124] cell__state = {l = , i = } [18:12:58.124] path = "\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000@\a\272\312\376\177\000\000\360\006\272\312\376\177\000\000\001\000\000\000\000\000\000\000\004\000\000\t", '\000' , "\304\004\272\312\376\177\000\000X\300\001kAV\000\000Q\000\222kAV\000\000\300\000\222kAV\000\000\240\037\000\000\377\377\002\000g\300\001kAV\000\000\300\000\222kAV\000\000\002\000\000\000\000\000\000\000\352\036\335jAV\000\000\346\004\272\312\376\177\000\000\344\004\272\312\376\177", '\000' , "\060\060\060\030\000\000\000\060\000\000\000\300\004\272\312\376\177\000\000\303\037\335jAV\000\000\000\000\000\000\000\000\000\000"... [18:12:58.124] cell = [18:12:58.124] fd = [18:12:58.124] tles = 0x56416d9d6de8 [18:12:58.124] __func__ = "XLogFileReadAnyTLI" [18:12:58.124] #8 0x000056416a4a5c5c in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=100683560, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=100683536, replayTLI=1, replayLSN=100683536, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3770 [18:12:58.124] oldSource = [18:12:58.124] startWalReceiver = false [18:12:58.124] last_fail_time = 763668584618472 [18:12:58.124] now = [18:12:58.124] streaming_reply_sent = true [18:12:58.124] __func__ = "WaitForWALToBecomeAvailable" [18:12:58.124] #9 0x000056416a4a6791 in XLogPageRead (xlogreader=0x56416d971f80, targetPagePtr=100679680, reqLen=3880, targetRecPtr=100683536, readBuf=0x56416d9c0e08 "\024\321\006") at ../src/backend/access/transam/xlogrecovery.c:3351 [18:12:58.124] private = 0x56416d971f60 [18:12:58.124] emode = 15 [18:12:58.124] targetPageOff = 16384 [18:12:58.124] targetSegNo = 6 [18:12:58.124] r = [18:12:58.124] __func__ = "XLogPageRead" [18:12:58.124] #10 0x000056416a4aebec in ReadPageInternal (state=state@entry=0x56416d971f80, pageptr=pageptr@entry=100679680, reqLen=reqLen@entry=3880) at ../src/backend/access/transam/xlogreader.c:1060 [18:12:58.124] readLen = [18:12:58.124] targetPageOff = 16384 [18:12:58.124] targetSegNo = 6 [18:12:58.124] hdr = [18:12:58.124] #11 0x000056416a4b0847 in XLogDecodeNextRecord (state=state@entry=0x56416d971f80, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:597 [18:12:58.124] RecPtr = 100683536 [18:12:58.124] record = [18:12:58.124] targetPagePtr = 100679680 [18:12:58.124] randAccess = false [18:12:58.124] len = [18:12:58.124] total_len = [18:12:58.124] targetRecOff = 3856 [18:12:58.124] pageHeaderSize = [18:12:58.124] assembled = false [18:12:58.124] gotheader = [18:12:58.124] readOff = [18:12:58.124] decoded = 0x0 [18:12:58.124] errormsg = 0x56416b00a5e7 "LOG" [18:12:58.124] #12 0x000056416a4b13e9 in XLogReadAhead (state=0x56416d971f80, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:973 [18:12:58.124] result = [18:12:58.124] #13 0x000056416a49bd2b in XLogPrefetcherNextBlock (pgsr_private=94839011487152, lsn=0x56416d9d6788) at ../src/backend/access/transam/xlogprefetcher.c:492 [18:12:58.124] nonblocking = [18:12:58.124] record = [18:12:58.124] prefetcher = 0x56416d972db0 [18:12:58.124] reader = 0x56416d971f80 [18:12:58.124] replaying_lsn = 100683488 [18:12:58.124] __func__ = "XLogPrefetcherNextBlock" [18:12:58.124] #14 0x000056416a4a00f4 in lrq_prefetch (lrq=) at ../src/backend/access/transam/xlogprefetcher.c:252 [18:12:58.124] No locals. [18:12:58.124] #15 lrq_complete_lsn (lsn=0, lrq=0x56416d9d66b8) at ../src/backend/access/transam/xlogprefetcher.c:290 [18:12:58.124] No locals. [18:12:58.124] #16 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x56416d972db0, errmsg=errmsg@entry=0x7ffecaba4a58) at ../src/backend/access/transam/xlogprefetcher.c:1037 [18:12:58.124] record = [18:12:58.124] replayed_up_to = 0 [18:12:58.124] #17 0x000056416a4a6fbe in ReadRecord (xlogprefetcher=0x56416d972db0, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=1) at ../src/backend/access/transam/xlogrecovery.c:3151 [18:12:58.124] errormsg = 0x56416d972790 "" [18:12:58.124] record = [18:12:58.124] xlogreader = 0x56416d971f80 [18:12:58.124] private = [18:12:58.143] __func__ = "ReadRecord" [18:12:58.143] #18 0x000056416a4aa155 in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1832 [18:12:58.143] xtime = [18:12:58.143] ru0 = {tv = {tv_sec = 1710353380, tv_usec = 91720}, ru = {ru_utime = {tv_sec = 0, tv_usec = 1658}, ru_stime = {tv_sec = 0, tv_usec = 0}, {ru_maxrss = 16004, __ru_maxrss_word = 16004}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 404, __ru_minflt_word = 404}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 4, __ru_nvcsw_word = 4}, {ru_nivcsw = 1, __ru_nivcsw_word = 1}}} [18:12:58.143] record = 0x7f7dc2157f70 [18:12:58.143] reachedRecoveryTarget = false [18:12:58.143] replayTLI = 1 [18:12:58.143] __func__ = "PerformWalRecovery" [18:12:58.143] #19 0x000056416a493e02 in StartupXLOG () at ../src/backend/access/transam/xlog.c:5723 [18:12:58.143] Insert = [18:12:58.143] checkPoint = {redo = 83886120, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 749}, nextOid = 24576, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1710353379, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 749} [18:12:58.143] wasShutdown = false [18:12:58.143] didCrash = [18:12:58.143] haveTblspcMap = false [18:12:58.143] haveBackupLabel = true [18:12:58.143] EndOfLog = [18:12:58.143] EndOfLogTLI = [18:12:58.143] newTLI = [18:12:58.143] performedWalRecovery = [18:12:58.143] endOfRecoveryInfo = [18:12:58.143] abortedRecPtr = [18:12:58.143] missingContrecPtr = [18:12:58.143] oldestActiveXID = [18:12:58.143] promoted = false [18:12:58.143] __func__ = "StartupXLOG" [18:12:58.143] #20 0x000056416a943f82 in StartupProcessMain () at ../src/backend/postmaster/startup.c:252 [18:12:58.143] No locals. [18:12:58.143] #21 0x000056416a93449b in AuxiliaryProcessMain (auxtype=auxtype@entry=B_STARTUP) at ../src/backend/postmaster/auxprocess.c:91 [18:12:58.143] __func__ = "AuxiliaryProcessMain" [18:12:58.143] #22 0x000056416a93f23c in StartChildProcess (type=type@entry=B_STARTUP) at ../src/backend/postmaster/postmaster.c:5336 [18:12:58.143] pid = [18:12:58.143] __func__ = "StartChildProcess" [18:12:58.143] #23 0x000056416a943710 in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x56416d970e40) at ../src/backend/postmaster/postmaster.c:1470 [18:12:58.143] opt = [18:12:58.143] status = [18:12:58.143] userDoption = [18:12:58.143] listen_addr_saved = false [18:12:58.143] output_config_variable = [18:12:58.143] __func__ = "PostmasterMain" [18:12:58.143] #24 0x000056416a76ea84 in main (argc=4, argv=0x56416d970e40) at ../src/backend/main/main.c:197 [18:12:58.143] do_check_root = [18:12:58.143] [18:12:58.143] Thread 1 (LWP 36172): [18:12:58.143] #0 0x00007f7dc4f06fc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [18:12:58.143] No symbol table info available. [18:12:58.143] #1 0x00007f7dc4fb813a in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [18:12:58.143] No symbol table info available. [18:12:58.143] #2 0x00007f7dc4fc9a6f in clone () from /lib/x86_64-linux-gnu/libc.so.6 [18:12:58.143] No symbol table info available.