The planning in my last article left a fair amount to the imagination, and in particular several pending decisions without sufficient detail to move discussion forward. The picture coming into view is that through a habit of insufficiently structured communications between engineering and management, I've been attempting to bear more weight of decision-making than is my due, trying to optimize things from the technical point of view which, while not unimportant, does not always align with the needs of a business to survive and grow in a dynamic market. I'll also take the opportunity to encourage present or future management to lean on me for support should it happen that I wander off into my own world.
With that in mind, let's expand a bit on the remaining tasks and decisions.
Switching to the now-builtin bignum support
I noted this as something that could potentially be deferred due to time constraints, because it does work as is. Unfortunately upon testing I've found that performance is noticeably degraded from what it used to be, at 14 seconds to derive an address from a private key on my dev system where I recall this having been closer to 5 seconds on earlier versions of the interpreter. (Transaction signing would be several times this.) I suspect a big part of the slowdown is the now-redundant layers of type and overflow checking; I'm unsure how to confirm this other than just trying the change. FWIW, I don't anticipate it taking all that long; maybe I take an hour and see if I can get it done.
Unspent output accounting; import of inbound or previously unknown outbound payments
The wallet tracks three types of data: private keys, the public keys and addresses derived from them, and transactions affecting its addresses. Queries such as balance and unspent outputs can be resolved by scanning the list of transactions, or tracked separately. As I see it, the "full scan" will have to work anyway so separate tracking, while desirable from an algorithmic scalability standpoint, would be a premature optimization at this stage. Without any such derived data to maintain, the "import" operation can simply mean appending lines containing encoded transaction data to a text file. Which brings us to:
External representation of transaction data (S-expression or maybe JSON)
I'm presently thinking S-exprs are the way to go; it keeps things simple on the Scheme side, and the Python side would only need to emit them, not parse them. What I'm not in a position to see is whether interoperability with other JSON-based tools would be desirable. I do have a parser in Scheme for a simplified form of JSON - omitting some Unicode details - that would suffice here.
Transaction input selection (menu-based?)
A possible cut here is to skip any sort of interface and use a simple selection strategy like FIFO. A manual interface could be added later. The idea here was to print a numbered listing of outputs by address/age/value and prompt for which ones to spend until sufficient value is accumulated.
ECDSA S-value normalization?
Turns out I've already got this: "As (r, -s mod n) is also a valid signature, the result is canonicalized to use the lesser of the two possible s-values." Thanks, 2017-me! (Some background.)
Fetching blocks from TRB (new RPC? Bitcoin protocol?)
In fact this could be done using the existing "dumpblock", perhaps using a named pipe. An RPC to return a block directly still seems preferable to me, but patching TRB isn't free. The "Bitcoin protocol" angle here meant connecting to port 8333 and using the p2p protocol commands to fetch a block, which is probably overkill if this is to be used in a batched or "pull" style.
Pushing raw transactions to TRB (likewise)
This one I don't think can be avoided.
Tracking watched addresses and synchronization state (SQL?)
The simple approach here, if indeed we're going for a batch model, would be a text file listing addresses and another that gives the last scanned block number/hash. When you run a scan, new transactions affecting the given addresses are emitted to a text file for transfer offline, and the last-scanned-block file is rewritten. If whatever data gets lost, you just rescan from an older block. Dealing with reorgs is where it could all get messier; one approach could be to insist on, say, 6 confirmations, and do a full rescan in the rare event of a deeper fork. How exactly one detects this and what could happen if one doesn't seems to warrant further consideration.
My first point to raise is about timeline expectations: the clients we agreed to deliver the wallet to by end of 2019 thought they'd be through in Nov. 2019. Travel has delayed some sessions and we have their final three in January 2020. These past 20 sessions since Sept have made or relationship with these clients much stronger compared to when we signed the contract - they are likely to be long time partners. Speaking with them this week verified the delivery of the code can be extended some time if investment makes sense.
With that being said, my next step is to recommend a re-read of http://trilema.com/2016/the-ideal-bitcoin-wallet/ and http://trilema.com/2016/how-to-cut-the-wallet/ . These articles inspired the original approach. You've implentented substantial code since. How does the current state of the code and design conform to these specs ? What gaps remain ? What questions arise now in reading these articles ? Let's go ahead and leave our questions as comments on those articles.
I know I need to refresh myself on aspects of the design choices and I will re-read these over the next days to sink in. Given that the original dec 13th deadline is flexible from the client's perspective, let's take the time we have to discuss and understand what approach correctly makes sense. I'm presently biased towards sufficiently extensive talks upfront and relatively less time coding than a brief terse exachange and longer than expected coding time. (Not implying a bias from you, just making mine explicit.)
Comment by Robinson Dorion — 2019-11-23 @ 02:40
"Speaking with them this week verified the delivery of the code can be extended some time if investment makes sense." - huh, that could change the shape of things quite a bit, but needs to be clarified ASAP because if I don't proceed pronto according to plan then I don't see it getting done in 2019. Are you saying you got a verbal agreement to a change? What is your understanding of the change, exactly? Or from another angle: by how long extended and how do we determine if investment makes sense?
I don't see extensive talks now as likely to save coding time in the context of getting what we have to a working state. If the goal is to build the ideal Republican thing, then yes, it reduces risk of having to redo things.
Independent of present clients, do you have a sense of what would be most helpful for those crucial early 2020 sales: existing & working cold-storage wallet, with some perhaps sub-optimal design choices, or more ideal thing under construction?
I'll revisit the articles now though.
Comment by Jacob Welsh — 2019-11-23 @ 04:35
1. On the timing, let's clarify in writing with them today.
2. Big num change to interpreter. Go ahead and invest the hour to try the change and report back status of either done or spec w/ time estimates for the remaining tasks.
3. On import to update state, append to text file encoded transaction data is fine for now.
4. I agree to start with S-expression. JSON interoperability looks like a nice to have at this point, it could be added at a later date, correct ?
5. On input selection, if there's to be no menu to start and no code to optimally select inputs, FIFO on the text file where the inputs can be manually ordered at the operators discretion looks like a reasonable first step and within the range of skills clients either already had or developed through working with us. I think working towards a menu based interface is a good long term strategy, but not a high priority at present.
7. On, ECDSA s value normalization, win :)
8. On Fetching blocks from trb, dump block with named pipe is preferred to trb patching atm.
9. On pushing raw transactions to trb, by unavoidable you mean a pushrawtransaction rpc call ?
10. On tracking watched addresses and synchronization state, I think hand cranked sync to update text files is most reasonable. Let operator set a cron job if he finds himself cranking more than he wants. Regarding reorgs, explaining Satoshi's 6 block confirmation as expectation and letting the underlying trb sort out the reorg is also reasonable.
Regarding sense for early 2020 sales, part of my approach in giving feedback here is to evaluate the usage within the context of the skills they either already had or are building through working with us and need for operation of similar tools, e.g. gpg.
Comment by Robinson Dorion — 2019-11-23 @ 15:20
4. Yes, though changing storage formats could be a pain for users.
9. Either a pushrawtransaction or connecting on port 8333, doing the version handshake and such to relay the transaction as if from another node. However the latter's more fragile e.g. if the node is running -nolisten or all inbound slots are in use, and the former's preferable per http://fr.anco.is/2017/01/removing-the-wallet-from-trb-first-thoughts#source_1 .
jfw: Otherwise: dorion is stuck on a fondleslate because the supposed campsite power supply didn't deliver. I'd let him be except that I have to clarify what wallet expectations actually are in light of http://fixpoint.welshcomputing.com/2019/next-steps-in-wallet-planning/#comment-42
diana_coman: jfw: so can you get hold of him to clarify on the wallet or not?
jfw: Yes but unreliably. I'm planning to email the clients, though they're traveling as well
diana_coman: tbh I'd much rather incline at this stage to *not* change the plans now but you do need to talk to him and agree if that's the best way forward atm or not.
jfw: aha, was my inclination too
diana_coman: is it perhaps a problem to have next year some "version 2" ?
jfw: no; though I think his concern is trying to get the plan right the first time so as not to have to redo things
diana_coman: eh, that he won't ~ever get, lol.
jfw: heh, and the v1 would be instructive in further planning I expect
diana_coman: quite.
diana_coman: it's more important to not get it catastrophically/fundamentally wrong, that's about it.
diana_coman: that said, there IS a cost to having clients change versions though
diana_coman: hence my question above whether this is a potential problem or to what extent
jfw: I'll add that to the list to clarify with them.
diana_coman: jfw: that is more to clarify with dorion really; they won't *know* to tell you exactly, how could they?
diana_coman: sure, you'll get an answer if you ask but ...
jfw: On reviewing the trilema wallet articles he linked, it looks like I've covered most of the points, except for things like better accounting functions and programmable input selection, which I'd think can be added later once better specified
diana_coman: so then it sounds like best to get v1 working really.
diana_coman: jfw: also, there is that thing: fewer questions to the clients, it's enough that they pay for it!!
diana_coman: lolz
jfw: "they won't *know* to tell you exactly" - indeed, I'll keep the pressure on him then :D
diana_coman: heh, have fun.
jfw: Do you have any thoughts on plain-text representation of structured data - S-expressions vs JSON and so on? (I wouldn't touch XML if I don't have to, lol)
jfw: Know if there's a preferred standard in Ada-land?
diana_coman: hm, I am not aware of an Ada standard as such; thinking of it now, I would expect it's not a huge headache to implement either really but I haven't spent the time to consider/compare why one or the other.
diana_coman: I suppose it depends on what you want you want to interface with further really
Comment by Jacob Welsh — 2019-11-23 @ 21:11
Diana Coman's pointed out some holes in my communication, so I'll make a fresh pass through the whole thing and then import the log.
About emailing them - if we do this, we ought to work out between us a more specific choice to propose i.e. *this* is what can be gained by extending the deadline. But on that, I'm not seeing much to be gained in usability in context of their skills. Getting an initial version out should provide more clarity from both the technical and business view on what if any additions would be desired in a v2. Any sort of rewrite is certainly not going to fit in January (not to imply you suggested it would).
On the articles and gaps - the present code and remaining plan is fundamentally in accord with them as far as I can see, with the exception of advanced accounting and optimal input selection, or other things that might be implicit under "better support code and integration generally". On input selection by ordering of a text file, I'm not certain yet how this works in practice because I'd been thinking of the data as a chronological list of transactions rather than reorderable list of unspent outputs. The latter could be extracted from the former on demand. Perhaps this is something that I could refine into a question for MP. In the categories of the "cutting the wallet" article, the design is most closely described by II, the solipsistic node: the offline part knows only about the transactions that affect it, with new outputs ("coinbases" there) relayed manually, thus is light on data storage yet has what it needs to create transactions.
Bignum change - I'll take that hour tomorrow.
"On import to update state, append to text file encoded transaction data is fine for now." - OK (note that this is based on the "chronological list of transactions" format).
"JSON interoperability" - per comment 4.
"I think working towards a menu based interface is a good long term strategy, but not a high priority at present." - ack.
"dump block with named pipe is preferred to trb patching atm." - ack.
"On pushing raw transactions to trb" - per comment 4.
"Let operator set a cron job if he finds himself cranking more than he wants." - you'd want to avoid a cron job rewriting text files while you're accessing them elsewhere; also I realized that the online component needs to track more state than previously mentioned, specifically the unspent outputs so that it knows when they're spent. Together these seem to me to argue in favor of a proper database, both regarding robustness and time, which I don't see being in conflict with the principle of plain-text for the offline part. As you know I'm pretty familiar with python/sqlite which is already in our stack.
Comment by Jacob Welsh — 2019-11-23 @ 23:58
diana_coman: jfw: ^
diana_coman: I'll be awk for a while.
jfw: ah sorry. Indeed, depends on what you want to connect it to. They can be easier to implement too if you pick a subset, of course that has its downsides too.
jfw: calls to mind the discussions of whether there can be one all-purpose OS, or all-purpose libraries for that matter, or every app has to be its own special thing
diana_coman: jfw: thing is, you are not really either aiming to or in a position to enforce a standard through this wallet implementation.
diana_coman: sure, do NOT go for xml! lolz
jfw: diana_coman: what would that imply here - that I should at least be picking an existing standard?
diana_coman: jfw: it implies that whatever you choose now may still need to change later if it turns out that the preferred/final standard is a different one
jfw: was I a pain again with the not acknowledging back there btw? perhaps still need to remind myself of the basic etiquette.
jfw: mm ok
diana_coman: jfw: I simply had no idea if you got/read that or not; it does make for a bit of funny stuff as it is a matter of basic feedback, yeah.
jfw: noted then.
diana_coman: jfw: but as far as I understood the situation now, the focus was on getting a *working* version that keeps to republican fundamentals and that hopefully you can further iterate upon as you gain experience with your biz.
jfw: that's right
diana_coman: fwiw, I'm also not that sure that dorion's point was re getting it perfect now or something, eg "sufficiently extensive talks upfront and relatively less time coding than a brief terse exachange and longer than expected coding time." - this reads to me more like making sure it's clear what is to implement and how long it will take ie clarifying not perfecting.
diana_coman: then again, you talked to him, not me.
diana_coman: jfw: put it also the other way: if you had this one extra month, what do you think would be delivered additionally?
diana_coman: give him the options: this month like this or in January like that.
jfw: I answered the "sufficiently extensive" point but he didn't quite clarify that I saw; perhaps I'm indeed over-interpreting.
diana_coman: sigh; /me goes to read your talk there.
jfw: what would be delivered additionally is something that's perhaps received more input from the wider republic - although it's possible that such feedback can't be accomodated by January, e.g. if it's to rewrite in Ada or something
diana_coman: jfw: that's not something feasible on that scale by end of January, no; and do NOT even think of re-writing it now because you can get an extension, GAH.
diana_coman: jfw: he said correctly " What gaps remain ? What questions arise now in reading these articles ? Let's go ahead and leave our questions as comments on those articles." - that's about as much additional feedback you can get and then see if it squeezes through in this implementation; so: any questions there?
diana_coman: jfw: tbh I think he stated there quite clearly for each point what next so not sure what is left other than the above ie if there are further gaps and/or questions on the fundamentals there
jfw: fair enough; I didn't see any big questions on the fundamentals, more on the details. Can check if there's anything that can be asked in the context of the articles.
diana_coman: and this "evaluate the usage within the context of the skills they either already had or are building through working with us" - so how well does this v1 fit what your clients learnt to use? to what extent would pushing the deadline to January increase this fit?
jfw: I don't see it increasing the fit, they'd be learning something new building on existing skills either way.
diana_coman: hm, I thought you two were better at talking to one another, huh.
jfw: we or at least I might be struggling a bit with the more async form
diana_coman: so then tell him all that ie 1. fundamentally it's aligned and fine, will check details today/tomorrow and see if there are further clarifications to ask for at those articles 2. not much to gain in extending the deadline as v1 of the wallet as it is would be a reasonable fit and either way they would still be learning something new (though ahem, do note that there are *degrees* there ie how much of a leap)
diana_coman: 3. can iterate in principle more usefully once this version is out and in use (if that's the idea).
diana_coman: jfw: I think I covered there main points but see if there's anything else you had in mind.
jfw: thank you diana_coman; I will review the points.
diana_coman: jfw: if you have something more concrete re Jan delivery, add that eg if we push the deadline to January, we can also have this and that that might/might not make it slightly/not at all/a lot easier for clients to learn/use/whatever.
diana_coman: jfw: and you know, reply to his points there eg will do etc.
jfw: diana_coman: will review & ack *all* the points then, right.
Comment by Jacob Welsh — 2019-11-24 @ 00:07
On my communication front, I expect to be back to stable internet and power tomorrow by noon. Reading this thread tonight, like what I see on first pass.
Comment by Robinson Dorion — 2019-11-24 @ 00:16
@ Jacob Welsh #4
> 4. Yes, though changing storage formats could be a pain for users.
I think the pain is tolerable for now.
> 9. Either a pushrawtransaction or connecting on port 8333, doing the version handshake and such to relay the transaction as if from another node. However the latter's more fragile e.g. if the node is running -nolisten or all inbound slots are in use, and the former's preferable per http://fr.anco.is/2017/01/removing-the-wallet-from-trb-first-thoughts#source_1 .
I think implementing the less fragile -- pushrawtransaction rpc -- as the higher priority. I take from your description that pushrawstransaction is also the simpler implementation, is that correct ?
> diana_coman: tbh I'd much rather incline at this stage to *not* change the plans now but you do need to talk to him and agree if that's the best way forward atm or not.
> jfw: aha, was my inclination too
> diana_coman: is it perhaps a problem to have next year some "version 2" ?
> jfw: no; though I think his concern is trying to get the plan right the first time so as not to have to redo things
> diana_coman: eh, that he won't ~ever get, lol.
> jfw: heh, and the v1 would be instructive in further planning I expect
> diana_coman: quite.
> diana_coman: it's more important to not get it catastrophically/fundamentally wrong, that's about it.
> diana_coman: that said, there IS a cost to having clients change versions though
> diana_coman: hence my question above whether this is a potential problem or to what extent
My main purpose in pointing out the potential flexibility is to identify it as an option, such that we don't box ourselves in from an external deadline that's not actually there. If we actually have a little extra time, to what extent is making use of it worth it ? Not saying we should change the schedule, pointing it out we can make it work if it's worth it.
> jfw: I'll add that to the list to clarify with them.
> diana_coman: jfw: that is more to clarify with dorion really; they won't *know* to tell you exactly, how could they?
> diana_coman: sure, you'll get an answer if you ask but ...
Yeah, let's come to a decision and then tell them rather than ask.
> jfw: On reviewing the trilema wallet articles he linked, it looks like I've covered most of the points, except for things like better accounting functions and programmable input selection, which I'd think can be added later once better specified
> diana_coman: so then it sounds like best to get v1 working really.
Agreed to both.
Comment by Robinson Dorion — 2019-11-25 @ 03:24
@Jacob Welsh
> Diana Coman's pointed out some holes in my communication, so I'll make a fresh pass through the whole thing and then import the log.
>
Thanks, this was helpful.
> About emailing them - if we do this, we ought to work out between us a more specific choice to propose i.e. *this* is what can be gained by extending the deadline. But on that, I'm not seeing much to be gained in usability in context of their skills. Getting an initial version out should provide more clarity from both the technical and business view on what if any additions would be desired in a v2. Any sort of rewrite is certainly not going to fit in January (not to imply you suggested it would).
>
Agreed on the communication with them.
Agreed that the foundations of what you have are within their skill set now and agree on getting the initial version out and no rewrite was implied really.
> On the articles and gaps - the present code and remaining plan is fundamentally in accord with them as far as I can see, with the exception of advanced accounting and optimal input selection, or other things that might be implicit under "better support code and integration generally". On input selection by ordering of a text file, I'm not certain yet how this works in practice because I'd been thinking of the data as a chronological list of transactions rather than reorderable list of unspent outputs. The latter could be extracted from the former on demand. Perhaps this is something that I could refine into a question for MP. In the categories of the "cutting the wallet" article, the design is most closely described by II, the solipsistic node: the offline part knows only about the transactions that affect it, with new outputs ("coinbases" there) relayed manually, thus is light on data storage yet has what it needs to create transactions.
Right, that confirms my read. I think the next step for v1 should be for you the write the specification for the wallet and publish it as an article here.
The spec article will : a) provide a basis to evaluate the difference between the planned functionality and usage for v1 and the code you currently have ; b) provide a basis for QA to work from and for user documentation to be written and c) help you refine the question for MP and prepared yourself with coherent context to link him to.
>
> Bignum change - I'll take that hour tomorrow.
>
Nice, how'd that turn out ?
> "On import to update state, append to text file encoded transaction data is fine for now." - OK (note that this is based on the "chronological list of transactions" format).
>
Ok and noted.
> "JSON interoperability" - per comment 4.
Is it agreed then v1 will be S-expression and not JSON ?
>
> "On pushing raw transactions to trb" - per comment 4.
>
> "Let operator set a cron job if he finds himself cranking more than he wants." - you'd want to avoid a cron job rewriting text files while you're accessing them elsewhere; also I realized that the online component needs to track more state than previously mentioned, specifically the unspent outputs so that it knows when they're spent. Together these seem to me to argue in favor of a proper database, both regarding robustness and time, which I don't see being in conflict with the principle of plain-text for the offline part. As you know I'm pretty familiar with python/sqlite which is already in our stack.
That's a good point about the cron job, hm.
So sqlite on online side and plain text on offline side, right ?
Comment by Robinson Dorion — 2019-11-25 @ 03:28
@ Jacob Welsh #6
> diana_coman: jfw: but as far as I understood the situation now, the focus was on getting a *working* version that keeps to republican fundamentals and that hopefully you can further iterate upon as you gain experience with your biz.
> jfw: that's right
>
Agreed.
> diana_coman: fwiw, I'm also not that sure that dorion's point was re getting it perfect now or something, eg "sufficiently extensive talks upfront and relatively less time coding than a brief terse exachange and longer than expected coding time." - this reads to me more like making sure it's clear what is to implement and how long it will take ie clarifying not perfecting.
> diana_coman: then again, you talked to him, not me.
> diana_coman: jfw: put it also the other way: if you had this one extra month, what do you think would be delivered additionally?
> diana_coman: give him the options: this month like this or in January like that.
> jfw: I answered the "sufficiently extensive" point but he didn't quite clarify that I saw; perhaps I'm indeed over-interpreting.
> diana_coman: sigh; /me goes to read your talk there.
>
My principal aim in sufficiently extensive talks is to clarify. It's been a long time since this wallet project was under development and I want to make sure I understand where it is and where it's going when. Was for sure, "If we have another month, what does that buy us?" Note: I don't ask that only from a code point of view. No one, clients included, wants to see Jacob kill himself to deliver on an unnecessary deadline when a couple weeks extension allows actual sustainable growth.
> jfw: what would be delivered additionally is something that's perhaps received more input from the wider republic - although it's possible that such feedback can't be accomodated by January, e.g. if it's to rewrite in Ada or something
> diana_coman: jfw: that's not something feasible on that scale by end of January, no; and do NOT even think of re-writing it now because you can get an extension, GAH.
> diana_coman: jfw: he said correctly " What gaps remain ? What questions arise now in reading these articles ? Let's go ahead and leave our questions as comments on those articles." - that's about as much additional feedback you can get and then see if it squeezes through in this implementation; so: any questions there?
> diana_coman: jfw: tbh I think he stated there quite clearly for each point what next so not sure what is left other than the above ie if there are further gaps and/or questions on the fundamentals there
> jfw: fair enough; I didn't see any big questions on the fundamentals, more on the details. Can check if there's anything that can be asked in the context of the articles.
>
> diana_coman: and this "evaluate the usage within the context of the skills they either already had or are building through working with us" - so how well does this v1 fit what your clients learnt to use? to what extent would pushing the deadline to January increase this fit?
> jfw: I don't see it increasing the fit, they'd be learning something new building on existing skills either way.
>
> diana_coman: hm, I thought you two were better at talking to one another, huh.
> jfw: we or at least I might be struggling a bit with the more async form
>
One thing to consider is, with extra time, how much better can/will the documentation be ? What is the current plan for documentation ? yrc and Gales Linux come with well written plain text documentation. You didn't have a blog when you published those. Now that you have a blog, does an FFA/Eucrypt style article series of the vpathes make the most sense ?
> diana_coman: so then tell him all that ie 1. fundamentally it's aligned and fine, will check details today/tomorrow and see if there are further clarifications to ask for at those articles 2. not much to gain in extending the deadline as v1 of the wallet as it is would be a reasonable fit and either way they would still be learning something new (though ahem, do note that there are *degrees* there ie how much of a leap)
> diana_coman: 3. can iterate in principle more usefully once this version is out and in use (if that's the idea).
> diana_coman: jfw: I think I covered there main points but see if there's anything else you had in mind.
> jfw: thank you diana_coman; I will review the points.
> diana_coman: jfw: if you have something more concrete re Jan delivery, add that eg if we push the deadline to January, we can also have this and that that might/might not make it slightly/not at all/a lot easier for clients to learn/use/whatever.
> diana_coman: jfw: and you know, reply to his points there eg will do etc.
> jfw: diana_coman: will review & ack *all* the points then, right.
Thanks for covering all the points, I think I covered all yours here.
At this point I see the next steps being an article to specify wallet functionality and usage and deciding if article series publication style is the best distribution strategy. Do you agree those are priorities ?
The async comms were difficult this week, but I think it's been productive. Back near stable power and connection so comms to normalize.
Comment by Robinson Dorion — 2019-11-25 @ 03:44
[...] More wallet planning and coaching thereon. [...]
Pingback by JFW review, week of Nov 18 2019 « Young Hands Club — 2019-11-25 @ 05:24
"I think the pain is tolerable for now." - I agree, if it turns out necessary.
"I think implementing the less fragile -- pushrawtransaction rpc -- as the higher priority. I take from your description that pushrawstransaction is also the simpler implementation, is that correct ?" - I think it would be the simpler implementation overall, though noting that pushrawtransaction would be a TRB patch whereas peer connection would be independent. I intend to ask trinque about this too: perhaps there's an existing solution by now.
"Not saying we should change the schedule, pointing it out we can make it work if it's worth it." - ok. Another opportunity cost of extending the schedule now could be the OS project, as I'm sure you're weighing.
"The spec article will : a) provide a basis to evaluate the difference between the planned functionality and usage for v1 and the code you currently have ;" - hmm, I thought this was done in what I already wrote. Could you clarify what you see as missing there?
"b) provide a basis for QA to work from and for user documentation to be written" - maybe this is some old folly on my part but... how does one test or document something that doesn't yet exist? These sorts of documents cover details that are not yet certain; and it's not like these tasks are going to be done in parallel by someone other than me. (Right?)
"c) help you refine the question for MP and prepared yourself with coherent context to link him to." - likewise, do you see incoherence in the last article? The main missing point I see is the data structure detail, which is what the question concerns.
I appreciate the effort to get the plans crystal clear; though I wonder if it's a luxurious level of consideration that we can't afford right now. I do at least owe an update on tasks and time estimates and perhaps this will clarify.
"Bignum change" - unfortunately I found it is not a quick job to switch to the new builtin operators. I wrote those to the Scheme standard; like Ada, this omits bit shifts, and my extensions for that only covered fixnums. There are more details included in the old bignum code that would need porting for the new builtins, though shifting looks like the major one at first glance.
Using those fixnum extensions I was able to replace some quotient/remainder ops in the older bignum code with bitwise ops, which gave a 12% speedup in address generation. Further optimizations are possible but would require a deeper dig from what I'm seeing.
One (unsurprising) lesson here is that when performance counts, language support needs to map well to what the hardware actually does.
"Is it agreed then v1 will be S-expression and not JSON ?" - Yep.
"So sqlite on online side and plain text on offline side, right ?" - Right.
"No one, clients included, wants to see Jacob kill himself to deliver on an unnecessary deadline when a couple weeks extension allows actual sustainable growth." - Thank you; pretty sure the extra month is not needed to buy my life, as long as the plan is able to proceed!
You can use <blockquote> in comments btw; might be simpler than all those >'s.
"What is the current plan for documentation ?" - I plan to write user-level documentation; this should at minimum be included as a text file with the code, so you can read offline on the wallet machine. I don't see a source code level dissection being at all required for this delivery, do you? Nice to have once the code is well stabilized. Can we make that decision once we get there?
"Do you agree those are priorities ?" - should be covered by the above.
Comment by Jacob Welsh — 2019-11-26 @ 00:39
cool.
Ok, report back here what trinque says.
At present I think TRB vpatch remains the way to go. One should be able to independently push a transaction.
Right, that changed the present situation.
My reply there : please clarify, what, if any, revisions are needed. I'm bias to say it makes sense to make a new article for clarity's sake.
I know there are some details to be worked out, my thinking is write the spec as far as you know it today and update it as it clarifies. I need to stay with you in parallel and you need to describe the status and choices when you run into a business decision. Seems to me the spec is a central document to maintain coherence for both ourselves and who ever else wants to contribute.
Things have changed since we started working on this, the update on tasks and time estimates will certainly clarify. I look forward to it.
Is this in a stable state for v1, or what is the next steps there/what remains for it to be ready for v1 ?
Aye.
Cool, as long as we know what the plan is and correctly maintain expectations for delivery with them.
how's this version read to you ?
Ok, so I'll have a plan with time estimates/due dates to read and reports as the tasks are completed.
Then a source code level dissection as part of v2 when we get there.
So source code level dissection article series is off in this case, and spec is either original article with updates where revisions needed or a new article with revisions included.
Comment by Robinson Dorion — 2019-11-26 @ 23:00
"report back here what trinque says." - so far nothing; perhaps it's time to try #a or #t.
"I'm bias to say it makes sense to make a new article for clarity's sake." - Done and let me know if it satisfies.
[re bignum] "what remains for it to be ready for v1 ?" - it works; just might take a couple minutes of CPU to sign transactions. Still beats doing 256-bit division by hand, right??
"how's this version read to you ?" - Stylin'!
"reports as the tasks are completed." - Will IRC be acceptable to you for this given the now relatively detailed task breakdown?
Comment by Jacob Welsh — 2019-11-28 @ 05:13
Some tentatively good news:
The patch will need some scrutiny but isn't huge.
Comment by Jacob Welsh — 2019-12-02 @ 06:54
[...] Next steps in wallet planning; [...]
Pingback by Draft gbw-node schema « Fixpoint — 2019-12-16 @ 22:37
[...] Next steps in planning. [...]
Pingback by Gales Bitcoin Wallet (re)release « Fixpoint — 2021-12-03 @ 08:54