Day changed to 2021-12-03
[09:05] jfw: http://fixpoint.welshcomputing.com/2021/gales-bitcoin-wallet-rerelease/
Day changed to 2021-12-07
[02:05] jfw: dorion: here's that magic number I mentioned whereby bitcoind checks the last 2500 blocks on disk at startup (actually a sneaky 2501 because of the less-than), unless given the undocumented -checkblocks in which case it checks all
[02:07] jfw: on my present machine this accounts for about 80% of the node startup time, 4x more than actually loading the full block index into RAM.
[02:13] jfw: it seems an excessive choice, if a choice has to be made - and it seems to me that it does. obviously we want to check at least the last block, as that's the most likely point of corruption in a crash. obviously it's obnoxious to check all of them unless explicitly requested - checking data integrity is all well and good but doesn't make much sense to bind it to the startup process, that's mixing
[02:13] jfw: two different things.
[02:16] jfw: but checking *only* the last block would strike me as imprudent, when at least a small handful could be done at minimal cost.
[02:18] jfw: one could argue for 2500 on the basis that it gets you one past the current blk####.dat file - but one would be wrong, if the blocks are <1 MB. so I really don't see it being substantially better than, say, 144 (a day's worth)
[02:20] jfw: dorion: any thoughts on what the number should be, if it should be changed at all, or some other approach?
[02:25] jfw: I couldn't figure out exactly why it treats the ReadFromDisk() and CheckBlock() failures as separate cases, but decided I didn't want to mess with it because a ReadFromBlock failure is necessarily a corrupt block or OS-level I/O error which it might not be safe or sane to attempt to recover from, while apparently it thinks it knows how to recover in the CheckBlock failure case
[02:28] jfw: however in that recovery case ("LoadBlockIndex() : *** found bad block at %d"), it looks like it really needs to be updating the 2500 threshold so as not to subvert the check.
[02:37] jfw: that is: my read is that it will roll back up to ~2500 blocks in the event that whatever internal checks fail (other than claimed proof of work matching reality - if that fails then it just aborts), but if all 2500 were bad then it just stops digging and carries on as if it knew it found a
[02:37] jfw: good one.
[02:41] jfw: I'm unconvinced that rolling back is valid at all (wouldn't it leave the bad blocks on disk and their headers in the block index db?), but the contemplated change is easy and at least won't make it any worse.
[06:47] jfw: http://fixpoint.welshcomputing.com/2021/crashproofing-bitcoind-the-easy-and-the-not-so-easy/
Day changed to 2021-12-08
[01:03] dorion: jfw: http://fixpoint.welshcomputing.com/2021/crashproofing-bitcoind-the-easy-and-the-not-so-easy/#comment-1729
[01:04] dorion: pretty sure that covers the questions above too.
[03:16] jfw: dorion: it does. http://fixpoint.welshcomputing.com/2021/crashproofing-bitcoind-the-easy-and-the-not-so-easy/#comment-1733
[18:39] dorion: jfw, http://fixpoint.welshcomputing.com/2021/crashproofing-bitcoind-the-easy-and-the-not-so-easy/#comment-1747
[20:52] jfw: dorion, replied.
[22:08] dorion: jfw, replied.
Day changed to 2021-12-23
[21:10] jfw: dorion: https://digilent.com/shop/jtag-hs2-programming-cable/ - possible off-the-shelf alternative to the defunct Bus Blaster; looks like xc3sprog already has a "cable db" entry to recognize it and it uses the same FTDI chip for usb->jtag that everyone does; might just need a physical adapter for the FUCKGOATS pinout/spacign
[21:11] jfw: mildly pricier but more streamlined than the BB.
[21:32] jfw: as for the bb's "open hardware" artifacts, they look possibly usable, with a couple blanks to fill in, provided we don't need to make any changes. That "gerber" format I'm finding is treated as a sort of dirty laundry of the PCB industry, used for historical reasons as a common interchange format for the board outlines but at arm's length, PDF style, and god help you if you want to edit one.
[21:35] jfw: whereas on the "source" end they locked it into the "eagle" turd. I tried 6 different "linux" versions, only one of which ran at all, and that one immediately demanded genuflections to the lord high Autodesk.
[21:41] jfw: the FG has no such problems because S.NSA simply... didn't publish any such sources, or even a bill of materials afaik, so we're stuck redoing it in any case.
Day changed to 2021-12-24
[00:01] jfw: dorion: "suckers who'll pay years worth of self-driving functionality that they never actually get" - the "pay" perhaps missing a "for"? otherwise it doesn't quite parse for me (how does one pay functionality?)
[00:02] jfw: and a couple other minor ones that more readily ECC-corrected.
[00:03] jfw: wondering why the links in the log excerpts aren't clickable; I thought mp-wp did that automatically.
[01:04] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3106 -- thanks, fixed.
[01:04] sourcerer: 2021-12-24 00:01:21 (#jwrd) jfw: dorion: "suckers who'll pay years worth of self-driving functionality that they never actually get" - the "pay" perhaps missing a "for"? otherwise it doesn't quite parse for me (how does one pay functionality?)
[18:22] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3101 -- nice find. question : "the same FTDI chip for usb->jtag that everyone does," what's the name, Xilinx ?
[18:22] sourcerer: 2021-12-23 21:10:19 (#jwrd) jfw: dorion: https://digilent.com/shop/jtag-hs2-programming-cable/ - possible off-the-shelf alternative to the defunct Bus Blaster; looks like xc3sprog already has a "cable db" entry to recognize it and it uses the same FTDI chip for usb->jtag that everyone does; might just need a physical adapter for the FUCKGOATS pinout/spacign
[18:27] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3103 -- good to read re using bb's open hardware. one strength of the bb and dp generally is certainly documentation.
[18:27] sourcerer: 2021-12-23 21:32:15 (#jwrd) jfw: as for the bb's "open hardware" artifacts, they look possibly usable, with a couple blanks to fill in, provided we don't need to make any changes. That "gerber" format I'm finding is treated as a sort of dirty laundry of the PCB industry, used for historical reasons as a common interchange format for the board outlines but at arm's length, PDF style, and god help you if you want to edit one.
[18:28] dorion: what are the alternatives to the gerber format ?
[18:32] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3104 -- are you talking about trying with the bus blaster ?
[18:32] sourcerer: 2021-12-23 21:35:15 (#jwrd) jfw: whereas on the "source" end they locked it into the "eagle" turd. I tried 6 different "linux" versions, only one of which ran at all, and that one immediately demanded genuflections to the lord high Autodesk.
[18:33] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3105 -- alright.
[18:33] sourcerer: 2021-12-23 21:41:44 (#jwrd) jfw: the FG has no such problems because S.NSA simply... didn't publish any such sources, or even a bill of materials afaik, so we're stuck redoing it in any case.
[18:35] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3108 -- yeah, I'm not sure. links are certainly working in the mp-wp logger.
[18:35] sourcerer: 2021-12-24 00:03:41 (#jwrd) jfw: wondering why the links in the log excerpts aren't clickable; I thought mp-wp did that automatically.
[20:59] jfw: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3111 - name of what? I'll try to clarify the actors here: Xilinx produces the CPLD chips (basically mini-FPGAs) that we're trying to program; JTAG is a protocol family, most generally for IC and PCB test automation but in this case Xilinx adopted it for reading/writing the logic configuration (bitstream) to the chip. FTDI makes widely
[20:59] sourcerer: 2021-12-24 18:22:37 (#jwrd) dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3101 -- nice find. question : "the same FTDI chip for usb->jtag that everyone does," what's the name, Xilinx ?
[20:59] jfw: used USB interface chips such as seen in serial converters and this and other JTAG adapters including the bus blaster itself.
[21:05] jfw: as to the digilent cable, turns out it has a manual that they just couldn't bother to link from the sales page, from which I'm gathering that it should plug into the FG's 6-pin/.1" JTAG header, no adapter or finnicky broken-out jumper wires necessary.
[21:12] jfw: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3113 - I wonder if you got an optimistic picture there; what I was basically saying was that myeah, the 'open source' aspect doesn't quite hold up in practice; specifically it requires you to have already sold your soul to do anything meaningful with the 'source'.
[21:12] sourcerer: 2021-12-24 18:27:20 (#jwrd) dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3103 -- good to read re using bb's open hardware. one strength of the bb and dp generally is certainly documentation.
[21:15] jfw: re alternatives to gerber, I suppose it depends on what for; in general it would come down to what each pcb house can accept as input for their automated manufacturing tools. gerber is at least nice in that it's an open standard that everyone supports.
[21:18] jfw: my critique was more about the *way* people in the industry use it as evidenced by what their tools do & don't do & emit.
[21:20] jfw: re Eagle, I had in mind to try loading up the bb's provided schematic & board layout files, yes.
[21:26] jfw: and I might as well expand on 'genuflections to the lord high Autodesk': it didn't ask for a license key, as might be expected of a proprietary package; rather, it loaded what was evidently an internal web browser - bereft of all the controls, like the popup ad windows of yore - wherein it asked for an Autodesk account login, as would be expected of a SaaS Platform.
[21:30] jfw: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3120 - in the logger's case it's logbot.awk that adds the formatting. perhaps mpwp doesn't linkify within a blockquote or something.
[21:30] sourcerer: 2021-12-24 18:35:04 (#jwrd) dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3108 -- yeah, I'm not sure. links are certainly working in the mp-wp logger.
Day changed to 2021-12-27
[23:00] jfw: http://fixpoint.welshcomputing.com/2021/header-only-archive-views-in-mp-wp/
Day changed to 2021-12-28
[14:30] dorion: jfw, nice. applied cleanly to mine.
[14:32] dorion: cleaning out my own manual feedbot queue : http://dorion-mode.com/2021/12/open-you-blind-eyes-with-cli-literacy/
[14:32] dorion: http://dorion-mode.com/2021/12/so-where-does-peter-schiff-get-it-twisted-with-regard-to-money-generally-and-bitcoin-in-particular/
[14:32] dorion: http://dorion-mode.com/2021/12/event-identity-money-and-banking-in-the-internet-age/
[09:05] jfw: http://fixpoint.welshcomputing.com/2021/gales-bitcoin-wallet-rerelease/
Day changed to 2021-12-07
[02:05] jfw: dorion: here's that magic number I mentioned whereby bitcoind checks the last 2500 blocks on disk at startup (actually a sneaky 2501 because of the less-than), unless given the undocumented -checkblocks in which case it checks all
[02:07] jfw: on my present machine this accounts for about 80% of the node startup time, 4x more than actually loading the full block index into RAM.
[02:13] jfw: it seems an excessive choice, if a choice has to be made - and it seems to me that it does. obviously we want to check at least the last block, as that's the most likely point of corruption in a crash. obviously it's obnoxious to check all of them unless explicitly requested - checking data integrity is all well and good but doesn't make much sense to bind it to the startup process, that's mixing
[02:13] jfw: two different things.
[02:16] jfw: but checking *only* the last block would strike me as imprudent, when at least a small handful could be done at minimal cost.
[02:18] jfw: one could argue for 2500 on the basis that it gets you one past the current blk####.dat file - but one would be wrong, if the blocks are <1 MB. so I really don't see it being substantially better than, say, 144 (a day's worth)
[02:20] jfw: dorion: any thoughts on what the number should be, if it should be changed at all, or some other approach?
[02:25] jfw: I couldn't figure out exactly why it treats the ReadFromDisk() and CheckBlock() failures as separate cases, but decided I didn't want to mess with it because a ReadFromBlock failure is necessarily a corrupt block or OS-level I/O error which it might not be safe or sane to attempt to recover from, while apparently it thinks it knows how to recover in the CheckBlock failure case
[02:28] jfw: however in that recovery case ("LoadBlockIndex() : *** found bad block at %d"), it looks like it really needs to be updating the 2500 threshold so as not to subvert the check.
[02:37] jfw: that is: my read is that it will roll back up to ~2500 blocks in the event that whatever internal checks fail (other than claimed proof of work matching reality - if that fails then it just aborts), but if all 2500 were bad then it just stops digging and carries on as if it knew it found a
[02:37] jfw: good one.
[02:41] jfw: I'm unconvinced that rolling back is valid at all (wouldn't it leave the bad blocks on disk and their headers in the block index db?), but the contemplated change is easy and at least won't make it any worse.
[06:47] jfw: http://fixpoint.welshcomputing.com/2021/crashproofing-bitcoind-the-easy-and-the-not-so-easy/
Day changed to 2021-12-08
[01:03] dorion: jfw: http://fixpoint.welshcomputing.com/2021/crashproofing-bitcoind-the-easy-and-the-not-so-easy/#comment-1729
[01:04] dorion: pretty sure that covers the questions above too.
[03:16] jfw: dorion: it does. http://fixpoint.welshcomputing.com/2021/crashproofing-bitcoind-the-easy-and-the-not-so-easy/#comment-1733
[18:39] dorion: jfw, http://fixpoint.welshcomputing.com/2021/crashproofing-bitcoind-the-easy-and-the-not-so-easy/#comment-1747
[20:52] jfw: dorion, replied.
[22:08] dorion: jfw, replied.
Day changed to 2021-12-23
[21:10] jfw: dorion: https://digilent.com/shop/jtag-hs2-programming-cable/ - possible off-the-shelf alternative to the defunct Bus Blaster; looks like xc3sprog already has a "cable db" entry to recognize it and it uses the same FTDI chip for usb->jtag that everyone does; might just need a physical adapter for the FUCKGOATS pinout/spacign
[21:11] jfw: mildly pricier but more streamlined than the BB.
[21:32] jfw: as for the bb's "open hardware" artifacts, they look possibly usable, with a couple blanks to fill in, provided we don't need to make any changes. That "gerber" format I'm finding is treated as a sort of dirty laundry of the PCB industry, used for historical reasons as a common interchange format for the board outlines but at arm's length, PDF style, and god help you if you want to edit one.
[21:35] jfw: whereas on the "source" end they locked it into the "eagle" turd. I tried 6 different "linux" versions, only one of which ran at all, and that one immediately demanded genuflections to the lord high Autodesk.
[21:41] jfw: the FG has no such problems because S.NSA simply... didn't publish any such sources, or even a bill of materials afaik, so we're stuck redoing it in any case.
Day changed to 2021-12-24
[00:01] jfw: dorion: "suckers who'll pay years worth of self-driving functionality that they never actually get" - the "pay" perhaps missing a "for"? otherwise it doesn't quite parse for me (how does one pay functionality?)
[00:02] jfw: and a couple other minor ones that more readily ECC-corrected.
[00:03] jfw: wondering why the links in the log excerpts aren't clickable; I thought mp-wp did that automatically.
[01:04] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3106 -- thanks, fixed.
[01:04] sourcerer: 2021-12-24 00:01:21 (#jwrd) jfw: dorion: "suckers who'll pay years worth of self-driving functionality that they never actually get" - the "pay" perhaps missing a "for"? otherwise it doesn't quite parse for me (how does one pay functionality?)
[18:22] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3101 -- nice find. question : "the same FTDI chip for usb->jtag that everyone does," what's the name, Xilinx ?
[18:22] sourcerer: 2021-12-23 21:10:19 (#jwrd) jfw: dorion: https://digilent.com/shop/jtag-hs2-programming-cable/ - possible off-the-shelf alternative to the defunct Bus Blaster; looks like xc3sprog already has a "cable db" entry to recognize it and it uses the same FTDI chip for usb->jtag that everyone does; might just need a physical adapter for the FUCKGOATS pinout/spacign
[18:27] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3103 -- good to read re using bb's open hardware. one strength of the bb and dp generally is certainly documentation.
[18:27] sourcerer: 2021-12-23 21:32:15 (#jwrd) jfw: as for the bb's "open hardware" artifacts, they look possibly usable, with a couple blanks to fill in, provided we don't need to make any changes. That "gerber" format I'm finding is treated as a sort of dirty laundry of the PCB industry, used for historical reasons as a common interchange format for the board outlines but at arm's length, PDF style, and god help you if you want to edit one.
[18:28] dorion: what are the alternatives to the gerber format ?
[18:32] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3104 -- are you talking about trying with the bus blaster ?
[18:32] sourcerer: 2021-12-23 21:35:15 (#jwrd) jfw: whereas on the "source" end they locked it into the "eagle" turd. I tried 6 different "linux" versions, only one of which ran at all, and that one immediately demanded genuflections to the lord high Autodesk.
[18:33] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3105 -- alright.
[18:33] sourcerer: 2021-12-23 21:41:44 (#jwrd) jfw: the FG has no such problems because S.NSA simply... didn't publish any such sources, or even a bill of materials afaik, so we're stuck redoing it in any case.
[18:35] dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3108 -- yeah, I'm not sure. links are certainly working in the mp-wp logger.
[18:35] sourcerer: 2021-12-24 00:03:41 (#jwrd) jfw: wondering why the links in the log excerpts aren't clickable; I thought mp-wp did that automatically.
[20:59] jfw: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3111 - name of what? I'll try to clarify the actors here: Xilinx produces the CPLD chips (basically mini-FPGAs) that we're trying to program; JTAG is a protocol family, most generally for IC and PCB test automation but in this case Xilinx adopted it for reading/writing the logic configuration (bitstream) to the chip. FTDI makes widely
[20:59] sourcerer: 2021-12-24 18:22:37 (#jwrd) dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3101 -- nice find. question : "the same FTDI chip for usb->jtag that everyone does," what's the name, Xilinx ?
[20:59] jfw: used USB interface chips such as seen in serial converters and this and other JTAG adapters including the bus blaster itself.
[21:05] jfw: as to the digilent cable, turns out it has a manual that they just couldn't bother to link from the sales page, from which I'm gathering that it should plug into the FG's 6-pin/.1" JTAG header, no adapter or finnicky broken-out jumper wires necessary.
[21:12] jfw: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3113 - I wonder if you got an optimistic picture there; what I was basically saying was that myeah, the 'open source' aspect doesn't quite hold up in practice; specifically it requires you to have already sold your soul to do anything meaningful with the 'source'.
[21:12] sourcerer: 2021-12-24 18:27:20 (#jwrd) dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3103 -- good to read re using bb's open hardware. one strength of the bb and dp generally is certainly documentation.
[21:15] jfw: re alternatives to gerber, I suppose it depends on what for; in general it would come down to what each pcb house can accept as input for their automated manufacturing tools. gerber is at least nice in that it's an open standard that everyone supports.
[21:18] jfw: my critique was more about the *way* people in the industry use it as evidenced by what their tools do & don't do & emit.
[21:20] jfw: re Eagle, I had in mind to try loading up the bb's provided schematic & board layout files, yes.
[21:26] jfw: and I might as well expand on 'genuflections to the lord high Autodesk': it didn't ask for a license key, as might be expected of a proprietary package; rather, it loaded what was evidently an internal web browser - bereft of all the controls, like the popup ad windows of yore - wherein it asked for an Autodesk account login, as would be expected of a SaaS Platform.
[21:30] jfw: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3120 - in the logger's case it's logbot.awk that adds the formatting. perhaps mpwp doesn't linkify within a blockquote or something.
[21:30] sourcerer: 2021-12-24 18:35:04 (#jwrd) dorion: http://fixpoint.welshcomputing.com/2021/jwrd-logs-for-Dec-2021/#3108 -- yeah, I'm not sure. links are certainly working in the mp-wp logger.
Day changed to 2021-12-27
[23:00] jfw: http://fixpoint.welshcomputing.com/2021/header-only-archive-views-in-mp-wp/
Day changed to 2021-12-28
[14:30] dorion: jfw, nice. applied cleanly to mine.
[14:32] dorion: cleaning out my own manual feedbot queue : http://dorion-mode.com/2021/12/open-you-blind-eyes-with-cli-literacy/
[14:32] dorion: http://dorion-mode.com/2021/12/so-where-does-peter-schiff-get-it-twisted-with-regard-to-money-generally-and-bitcoin-in-particular/
[14:32] dorion: http://dorion-mode.com/2021/12/event-identity-money-and-banking-in-the-internet-age/