[12:33:26.570] src/tools/ci/cores_backtrace.sh linux /tmp/cores [12:33:27.354] dumping /tmp/cores/postgres-3-26107.core for /tmp/cirrus-ci-build/build/tmp_install/usr/local/pgsql/bin/postgres [12:33:29.021] [New LWP 26107] [12:33:29.021] [New LWP 24814] [12:33:29.021] Downloading separate debug info for /lib/x86_64-linux-gnu/libm.so.6... [12:33:29.784] Downloading separate debug info for /lib/x86_64-linux-gnu/libdl.so.2... [12:33:30.546] Downloading separate debug info for /lib/x86_64-linux-gnu/librt.so.1... [12:33:31.769] Downloading separate debug info for /lib/x86_64-linux-gnu/libgssapi_krb5.so.2... [12:33:34.129] Downloading separate debug info for /lib/x86_64-linux-gnu/libicuuc.so.67... [12:33:34.129] Downloading separate debug info for /.cache/debuginfod_client/f871bbd529a02abd860f0d16b842b5b20234cb49/debuginfo... [12:33:36.001] Downloading separate debug info for /lib/x86_64-linux-gnu/libicui18n.so.67... [12:33:37.212] Downloading separate debug info for /lib/x86_64-linux-gnu/libldap_r-2.4.so.2... [12:33:38.789] Downloading separate debug info for /lib/x86_64-linux-gnu/libxml2.so.2... [12:33:39.852] Downloading separate debug info for /lib/x86_64-linux-gnu/liblz4.so.1... [12:33:40.618] Downloading separate debug info for /lib/x86_64-linux-gnu/libpam.so.0... [12:33:41.840] Downloading separate debug info for /lib/x86_64-linux-gnu/libssl.so.1.1... [12:33:43.448] Downloading separate debug info for /lib/x86_64-linux-gnu/libcrypto.so.1.1... [12:33:44.857] Downloading separate debug info for /lib/x86_64-linux-gnu/libsystemd.so.0... [12:33:45.767] Downloading separate debug info for /lib/x86_64-linux-gnu/libz.so.1... [12:33:47.299] Downloading separate debug info for /lib/x86_64-linux-gnu/libzstd.so.1... [12:33:48.687] Downloading separate debug info for /lib/x86_64-linux-gnu/libpthread.so.0... [12:33:49.156] [Thread debugging using libthread_db enabled] [12:33:49.156] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [12:33:50.219] Downloading separate debug info for /lib64/ld-linux-x86-64.so.2... [12:33:51.606] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5.so.3... [12:33:52.663] Downloading separate debug info for /lib/x86_64-linux-gnu/libk5crypto.so.3... [12:33:53.276] Downloading separate debug info for /lib/x86_64-linux-gnu/libcom_err.so.2... [12:33:54.039] Downloading separate debug info for /lib/x86_64-linux-gnu/libkrb5support.so.0... [12:33:56.315] Downloading separate debug info for /lib/x86_64-linux-gnu/libstdc++.so.6... [12:33:57.251] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcc_s.so.1... [12:33:58.015] Downloading separate debug info for /lib/x86_64-linux-gnu/liblber-2.4.so.2... [12:33:58.925] Downloading separate debug info for /lib/x86_64-linux-gnu/libresolv.so.2... [12:33:59.842] Downloading separate debug info for /lib/x86_64-linux-gnu/libsasl2.so.2... [12:34:01.429] Downloading separate debug info for /lib/x86_64-linux-gnu/libgnutls.so.30... [12:34:02.493] Downloading separate debug info for /lib/x86_64-linux-gnu/liblzma.so.5... [12:34:03.257] Downloading separate debug info for /lib/x86_64-linux-gnu/libaudit.so.1... [12:34:04.641] Downloading separate debug info for /lib/x86_64-linux-gnu/libgcrypt.so.20... [12:34:05.256] Downloading separate debug info for /lib/x86_64-linux-gnu/libkeyutils.so.1... [12:34:06.790] Downloading separate debug info for /lib/x86_64-linux-gnu/libp11-kit.so.0... [12:34:07.550] Downloading separate debug info for /lib/x86_64-linux-gnu/libidn2.so.0... [12:34:08.771] Downloading separate debug info for /lib/x86_64-linux-gnu/libunistring.so.2... [12:34:09.538] Downloading separate debug info for /lib/x86_64-linux-gnu/libtasn1.so.6... [12:34:10.609] Downloading separate debug info for /lib/x86_64-linux-gnu/libnettle.so.8... [12:34:11.530] Downloading separate debug info for /lib/x86_64-linux-gnu/libhogweed.so.6... [12:34:12.753] Downloading separate debug info for /lib/x86_64-linux-gnu/libgmp.so.10... [12:34:13.368] Downloading separate debug info for /lib/x86_64-linux-gnu/libcap-ng.so.0... [12:34:14.285] Downloading separate debug info for /lib/x86_64-linux-gnu/libgpg-error.so.0... [12:34:15.046] Downloading separate debug info for /lib/x86_64-linux-gnu/libffi.so.7... [12:34:15.537] Core was generated by `postgres: standby2: startup recovering 000000010000000000000006 '. [12:34:15.537] Program terminated with signal SIGQUIT, Quit. [12:34:15.537] #0 0x00007f8f96fadfc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [12:34:15.565] [Current thread is 1 (LWP 26107)] [12:34:15.681] [12:34:15.681] Thread 2 (Thread 0x7f8f94944a40 (LWP 24814)): [12:34:15.681] #0 0x00007f8f97070a61 in clone () from /lib/x86_64-linux-gnu/libc.so.6 [12:34:15.681] No symbol table info available. [12:34:15.681] #1 0x00007f8f9705ee5e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [12:34:15.681] No symbol table info available. [12:34:15.681] #2 0x00007f8f9705f497 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [12:34:15.681] No symbol table info available. [12:34:15.681] #3 0x00007f8f9705ecdb in posix_spawn () from /lib/x86_64-linux-gnu/libc.so.6 [12:34:15.681] No symbol table info available. [12:34:15.681] #4 0x00007f8f96fbaa69 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [12:34:15.681] No symbol table info available. [12:34:15.681] #5 0x0000560f7fefc4c6 in RestoreArchivedFile (path=path@entry=0x7fffbd2a4760 "", xlogfname=xlogfname@entry=0x7fffbd2a4720 "00000002.history", recovername=recovername@entry=0x560f80a0d9f0 "RECOVERYHISTORY", expectedSize=expectedSize@entry=0, cleanupEnabled=cleanupEnabled@entry=false) at ../src/backend/access/transam/xlogarchive.c:177 [12:34:15.681] xlogpath = "pg_wal/RECOVERYHISTORY\000\000\b\000\000\000\000\000\000\066\000\000\000\000\000\000\000\000\216\061\206\200\000\000\000\062\000\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000̡\240\200\017V\000\000\b\000\000\000\000\000\000\000\271G\240\200\017V\000\000HF*\275\377\177\000\000\265G\240\200\017V\000\000\272,\206\200\017V\000\000\301G\240\200\017V\000\000\271G\240\200\017V\000\000HF*\275\377\177\000\000\265G\240\200\017V\000\000\030I\224\224\217\177\000\000X6\206\200\017V", '\000' , "\b", '\000' , "\060", '\000' , "\002\000\000\000\000\000\000\000"... [12:34:15.681] xlogRestoreCmd = 0x560f82dac5e8 "cp \"/tmp/cirrus-ci-build/build/testrun/recovery/040_standby_failover_slots_sync/data/t_040_standby_failover_slots_sync_publisher_data/archives/00000002.history\" \"pg_wal/RECOVERYHISTORY\"" [12:34:15.681] lastRestartPointFname = '0' , "\000٠\200\017V\000\000\350A*\275\377\177\000\000\243٠\200\017V\000\000\030I\224\224\217\177\000\000\026\067\206\200\017V", '\000' , "\b", '\000' , "\060", '\000' , "\002", '\000' ... [12:34:15.681] rc = [12:34:15.681] stat_buf = {st_dev = 140254649796888, st_ino = 208314709782, st_nlink = 0, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 48, st_size = 5922071200885860208, st_blksize = 6366210006019162949, st_blocks = 4673356, st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 3891110078048108544, tv_nsec = 0}, st_ctim = {tv_sec = 0, tv_nsec = 0}, __glibc_reserved = {8, 94624582509004, 140736367051240}} [12:34:15.681] restartSegNo = [12:34:15.681] restartRedoPtr = 94624580776334 [12:34:15.681] restartTli = 0 [12:34:15.681] __func__ = "RestoreArchivedFile" [12:34:15.681] #6 0x0000560f7fed0eaf in existsTimeLineHistory (probeTLI=probeTLI@entry=2) at ../src/backend/access/transam/timeline.c:235 [12:34:15.681] path = '\000' , "\001\000\000\000\000\000\000\000ų\360\177\017V\000\000\034ɠ\200\017V\000\000\a\000\000\000\r\000\000\000\027ɠ\200\017V\000\000\030I\224\224\217\177\000\000PJ*\275\377\177\000\000\372>\206\200\017V\000\000\000\000\000\000\000\000\000\000\342\306@\201\017V\000\000`\307@\201\017V\000\000\024I*\275\377\177\000\000\070B\254\200\017V\000\000\361\306@\201\017V\000\000`\307@\201\017V\000\000\064I*\275\377\177\000\000GB\254\200\017V\000\000`\307@\201\017V\000\000\002\000\000\000\000\000\000\000\267F\206\200\017V\000\000\066I*\275\377\177\000\000\064I*\275\377\177", '\000' ... [12:34:15.681] histfname = "00000002.history", '\000' [12:34:15.681] fd = [12:34:15.681] __func__ = "existsTimeLineHistory" [12:34:15.681] #7 0x0000560f7fed0f9a in findNewestTimeLine (startTLI=startTLI@entry=1) at ../src/backend/access/transam/timeline.c:277 [12:34:15.681] newestTLI = 1 [12:34:15.681] probeTLI = 2 [12:34:15.681] #8 0x0000560f7ff07491 in rescanLatestTimeLine (replayTLI=replayTLI@entry=1, replayLSN=replayLSN@entry=100683512) at ../src/backend/access/transam/xlogrecovery.c:4113 [12:34:15.681] newExpectedTLEs = [12:34:15.681] found = [12:34:15.681] cell = [12:34:15.681] newtarget = [12:34:15.681] oldtarget = 1 [12:34:15.681] currentTle = 0x0 [12:34:15.681] __func__ = "rescanLatestTimeLine" [12:34:15.681] #9 0x0000560f7ff0c641 in WaitForWALToBecomeAvailable (RecPtr=RecPtr@entry=100683536, randAccess=false, fetching_ckpt=false, tliRecPtr=tliRecPtr@entry=100683512, replayTLI=1, replayLSN=100683512, nonblocking=false) at ../src/backend/access/transam/xlogrecovery.c:3674 [12:34:15.681] oldSource = XLOG_FROM_STREAM [12:34:15.681] startWalReceiver = false [12:34:15.681] last_fail_time = 768227460996735 [12:34:15.681] now = [12:34:15.681] streaming_reply_sent = true [12:34:15.681] __func__ = "WaitForWALToBecomeAvailable" [12:34:15.681] #10 0x0000560f7ff0d077 in XLogPageRead (xlogreader=0x560f82d59f30, targetPagePtr=100679680, reqLen=3856, targetRecPtr=100683512, readBuf=0x560f82da8db8 "\025\321\006") at ../src/backend/access/transam/xlogrecovery.c:3351 [12:34:15.681] private = 0x560f82d59f10 [12:34:15.681] emode = 15 [12:34:15.681] targetPageOff = 16384 [12:34:15.681] targetSegNo = 6 [12:34:15.681] r = [12:34:15.681] __func__ = "XLogPageRead" [12:34:15.681] #11 0x0000560f7ff154d2 in ReadPageInternal (state=state@entry=0x560f82d59f30, pageptr=pageptr@entry=100679680, reqLen=reqLen@entry=3856) at ../src/backend/access/transam/xlogreader.c:1060 [12:34:15.681] readLen = [12:34:15.681] targetPageOff = 16384 [12:34:15.681] targetSegNo = 6 [12:34:15.681] hdr = [12:34:15.681] #12 0x0000560f7ff1712d in XLogDecodeNextRecord (state=state@entry=0x560f82d59f30, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:597 [12:34:15.681] RecPtr = 100683512 [12:34:15.681] record = [12:34:15.681] targetPagePtr = 100679680 [12:34:15.681] randAccess = false [12:34:15.681] len = [12:34:15.681] total_len = [12:34:15.681] targetRecOff = 3832 [12:34:15.681] pageHeaderSize = [12:34:15.681] assembled = false [12:34:15.681] gotheader = [12:34:15.681] readOff = [12:34:15.681] decoded = 0x0 [12:34:15.681] errormsg = 0x560f80ab2581 "LOG" [12:34:15.681] #13 0x0000560f7ff17ccf in XLogReadAhead (state=0x560f82d59f30, nonblocking=nonblocking@entry=false) at ../src/backend/access/transam/xlogreader.c:973 [12:34:15.681] result = [12:34:15.681] #14 0x0000560f7ff02611 in XLogPrefetcherNextBlock (pgsr_private=94624619539808, lsn=0x560f82dbe738) at ../src/backend/access/transam/xlogprefetcher.c:492 [12:34:15.681] nonblocking = [12:34:15.681] record = [12:34:15.681] prefetcher = 0x560f82d5ad60 [12:34:15.681] reader = 0x560f82d59f30 [12:34:15.681] replaying_lsn = 100683464 [12:34:15.681] __func__ = "XLogPrefetcherNextBlock" [12:34:15.681] #15 0x0000560f7ff069da in lrq_prefetch (lrq=) at ../src/backend/access/transam/xlogprefetcher.c:252 [12:34:15.681] No locals. [12:34:15.681] #16 lrq_complete_lsn (lsn=0, lrq=0x560f82dbe668) at ../src/backend/access/transam/xlogprefetcher.c:290 [12:34:15.681] No locals. [12:34:15.681] #17 XLogPrefetcherReadRecord (prefetcher=prefetcher@entry=0x560f82d5ad60, errmsg=errmsg@entry=0x7fffbd2a8ea8) at ../src/backend/access/transam/xlogprefetcher.c:1037 [12:34:15.681] record = [12:34:15.681] replayed_up_to = 0 [12:34:15.681] #18 0x0000560f7ff0d8a4 in ReadRecord (xlogprefetcher=0x560f82d5ad60, emode=emode@entry=15, fetching_ckpt=fetching_ckpt@entry=false, replayTLI=1) at ../src/backend/access/transam/xlogrecovery.c:3151 [12:34:15.681] errormsg = 0x560f82d5a740 "" [12:34:15.681] record = [12:34:15.681] xlogreader = 0x560f82d59f30 [12:34:15.681] private = [12:34:15.681] __func__ = "ReadRecord" [12:34:15.681] #19 0x0000560f7ff10a3b in PerformWalRecovery () at ../src/backend/access/transam/xlogrecovery.c:1832 [12:34:15.681] xtime = [12:34:15.681] ru0 = {tv = {tv_sec = 1714912255, tv_usec = 326949}, ru = {ru_utime = {tv_sec = 0, tv_usec = 1437}, ru_stime = {tv_sec = 0, tv_usec = 0}, {ru_maxrss = 16232, __ru_maxrss_word = 16232}, {ru_ixrss = 0, __ru_ixrss_word = 0}, {ru_idrss = 0, __ru_idrss_word = 0}, {ru_isrss = 0, __ru_isrss_word = 0}, {ru_minflt = 406, __ru_minflt_word = 406}, {ru_majflt = 5, __ru_majflt_word = 5}, {ru_nswap = 0, __ru_nswap_word = 0}, {ru_inblock = 0, __ru_inblock_word = 0}, {ru_oublock = 0, __ru_oublock_word = 0}, {ru_msgsnd = 0, __ru_msgsnd_word = 0}, {ru_msgrcv = 0, __ru_msgrcv_word = 0}, {ru_nsignals = 0, __ru_nsignals_word = 0}, {ru_nvcsw = 4, __ru_nvcsw_word = 4}, {ru_nivcsw = 1, __ru_nivcsw_word = 1}}} [12:34:15.681] record = 0x7f8f941fef70 [12:34:15.681] reachedRecoveryTarget = false [12:34:15.699] replayTLI = 1 [12:34:15.699] __func__ = "PerformWalRecovery" [12:34:15.699] #20 0x0000560f7fefa45e in StartupXLOG () at ../src/backend/access/transam/xlog.c:5806 [12:34:15.699] Insert = [12:34:15.699] checkPoint = {redo = 83886120, ThisTimeLineID = 1, PrevTimeLineID = 1, fullPageWrites = true, nextXid = {value = 751}, nextOid = 24576, nextMulti = 1, nextMultiOffset = 0, oldestXid = 730, oldestXidDB = 1, oldestMulti = 1, oldestMultiDB = 1, time = 1714912254, oldestCommitTsXid = 0, newestCommitTsXid = 0, oldestActiveXid = 751} [12:34:15.699] wasShutdown = false [12:34:15.699] didCrash = [12:34:15.699] haveTblspcMap = false [12:34:15.699] haveBackupLabel = true [12:34:15.699] EndOfLog = [12:34:15.699] EndOfLogTLI = [12:34:15.699] newTLI = [12:34:15.699] performedWalRecovery = [12:34:15.699] endOfRecoveryInfo = [12:34:15.699] abortedRecPtr = [12:34:15.699] missingContrecPtr = [12:34:15.699] oldestActiveXID = [12:34:15.699] promoted = false [12:34:15.699] __func__ = "StartupXLOG" [12:34:15.699] #21 0x0000560f803c1beb in StartupProcessMain (startup_data=, startup_data_len=) at ../src/backend/postmaster/startup.c:258 [12:34:15.699] No locals. [12:34:15.699] #22 0x0000560f803b9207 in postmaster_child_launch (child_type=child_type@entry=B_STARTUP, startup_data=startup_data@entry=0x0, startup_data_len=startup_data_len@entry=0, client_sock=client_sock@entry=0x0) at ../src/backend/postmaster/launch_backend.c:265 [12:34:15.699] pid = [12:34:15.699] #23 0x0000560f803bb280 in StartChildProcess (type=type@entry=B_STARTUP) at ../src/backend/postmaster/postmaster.c:3928 [12:34:15.699] pid = [12:34:15.699] __func__ = "StartChildProcess" [12:34:15.699] #24 0x0000560f803c134a in PostmasterMain (argc=argc@entry=4, argv=argv@entry=0x560f82d58df0) at ../src/backend/postmaster/postmaster.c:1364 [12:34:15.699] opt = [12:34:15.699] status = [12:34:15.699] userDoption = [12:34:15.699] listen_addr_saved = false [12:34:15.699] output_config_variable = [12:34:15.699] __func__ = "PostmasterMain" [12:34:15.699] #25 0x0000560f801e1757 in main (argc=4, argv=0x560f82d58df0) at ../src/backend/main/main.c:197 [12:34:15.699] do_check_root = [12:34:15.699] [12:34:15.699] Thread 1 (LWP 26107): [12:34:15.699] #0 0x00007f8f96fadfc7 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6 [12:34:15.699] No symbol table info available. [12:34:15.699] #1 0x00007f8f9705f13a in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [12:34:15.699] No symbol table info available. [12:34:15.699] #2 0x00007f8f97070a6f in clone () from /lib/x86_64-linux-gnu/libc.so.6 [12:34:15.699] No symbol table info available.