Projects : gscm : gscm_genesis
| 1 | Testing & debugging after completion of rewrite: |
| 2 | |
| 3 | 1. Read/write ordering bug in pop() -- returned top of stack after having |
| 4 | already removed it. Needed to store to a temporary. |
| 5 | |
| 6 | 2. "return SC_NULL" instead of "RETURN(SC_NULL)" in reader. Considered renaming |
| 7 | the macro, but by any other name it would still refer to the same concept and |
| 8 | still be subject to confusion. Seems like a fundamental pun between C and the |
| 9 | embedded stack machine language. Wouldn't happen in Scheme since there return |
| 10 | is just something that happens to the last expresssion in a sequence, not a |
| 11 | command. |
| 12 | |
| 13 | 3. Toplevel error handler seems to be getting run without an actual error. |
| 14 | Replacing toplevel with more basic evaluation tests for now. |
| 15 | |
| 16 | 1 -> ERROR: apply: not a procedure: -945832504593831248 |
| 17 | |
| 18 | The error is correct (1 is not a procedure) but the display is wrong. |
| 19 | (Incorrect untagging?) |
| 20 | |
| 21 | Problem was in parse_integer / parse_decimal. Before, I used a C heap based |
| 22 | dynamic buffer mechanism, which tracked buffer pointer, allocation size and |
| 23 | filled size in a struct. In the rewrite I converted this to a string on the |
| 24 | Scheme heap, but since that doesn't distinguish fill from allocation, I |
| 25 | tracked fill in the lexeme_length global variable. (Ugly, but this one |
| 26 | internal use didn't seem to justify a whole new Scheme type.) But in |
| 27 | porting the integer parsers I used the Scheme string's length (allocation) |
| 28 | rather than the global. The chief hazard seems to be the two notions of |
| 29 | length, and passing around this un-truncated buffer in the reader internals |
| 30 | for efficiency. Reviewed other use of R_LEXEME for similar problems, |
| 31 | finding none. Changed the number parsers to take length as an argument to |
| 32 | at least reduce use of the global. |
| 33 | |
| 34 | #(1 2 3) -> ERROR: apply: not a procedure: #() |
| 35 | |
| 36 | Again, a correct error but bad read or print of the vector. |
| 37 | |
| 38 | In the reader, the length header of the vector was being set after |
| 39 | the "len" variable had already been used to count down to zero. |
| 40 | Rather than building the vector "by hand" I replaced this with the |
| 41 | recently added make_vector_uninit function. |
| 42 | |
| 43 | But this also revealed that the local variable "len" was being unsafely |
| 44 | used across a recursive call. Rather than push/popping it every time I took |
| 45 | it out of the loop to be computed later by list_length(). Except here we |
| 46 | already know the list is really a list, so I wrote a new unsafe_list_length |
| 47 | function to omit the type and cycle checks of list_length. |
| 48 | |
| 49 | (letrec () 1) -> ERROR: eval: empty combination |
| 50 | |
| 51 | LETREC was passing the body to EVAL_BODY through R_EXPR instead of R_BODY. |
| 52 | (Missed during manual register optimization.) Verified the other call to |
| 53 | EVAL_BODY. This fixed #3. |
| 54 | |
| 55 | 4. A register conflict in the error handler, where r_error_handler was assigned |
| 56 | to R_PROC, then R_ERR_DETAIL was referenced, but the latter two had been |
| 57 | optimized to share a register. Solved by reassigning R_ERR_DETAIL to a free |
| 58 | register. I did go to some pains to verify the register assignments but perhaps |
| 59 | gave short shrift to the error handler. It would be a good exercise to do a |
| 60 | full audit for register conflicts, once the code is a bit more settled. The |
| 61 | rules for this are detailed in a comment. |
| 62 | |
| 63 | 5. Two bugs relating to list_length, which is supposed to return -1 for safely |
| 64 | detecting non-lists. First, builtin_list_length stored its result in a "value" |
| 65 | instead of a "long", which being an unsigned type, prevented recognizing the |
| 66 | negative. Second, for odd length improper lists (e.g. (1 2 3 . 4)), the is_pair |
| 67 | check after the first of the two "fast = cdr(fast)" pointer advancements in the |
| 68 | loop returned directly, bypassing the "fast != SC_NULL" improper list check |
| 69 | after the loop. This irregularity resulted from being overly preoccupied with |
| 70 | (premature) hand-optimization (doing a single length += 2 after the two |
| 71 | advances in each loop instead of a length++ after each one), a tendency I've |
| 72 | had to fight in other places as well. (Chiefly, using the friendly type |
| 73 | constructor and accessor functions, which tag/untag values as needed, rather |
| 74 | than duplicating their functionality to save a few instructions on redundant |
| 75 | tag operations. In this, it helped that I convinced myself, by examining the |
| 76 | generated assembly, that the compiler optimizations really are smart enough to |
| 77 | factor out the tag operations, at least in some cases, and at least with static |
| 78 | functions.) |
| 79 | |
| 80 | 6. In SET, checked for cdr(R_EXPR) == SC_NULL after having overwritten the |
| 81 | intended R_EXPR with car(R_EXPR). Swapped the order. (Discovered by running the |
| 82 | "bad" semantic test set, but woulda come up soon enough anyway as it completely |
| 83 | broke "set!".) |
| 84 | |
| 85 | (Later... somewhere around Jan 2017) |
| 86 | |
| 87 | 7. yes '(' | gscm -> inf loop between gc/error handler for out of memory. |
| 88 | sc_error1 does a string() allocation to set R_ERR_MSG, but the stack doesn't |
| 89 | get dropped until after the longjmp. Fixed by dropping stack in the error |
| 90 | handler. Also wrote a toplevel error handler callable from Scheme implementing |
| 91 | the same behavior using a captured continuation. |
| 92 | |
| 93 | 8. The "apply" builtin did not copy its final argument (a list). The standard |
| 94 | doesn't seem to specify whether it should or not, but "list" is required to |
| 95 | return a newly allocated list, and my no-op implementation was assuming its |
| 96 | argument list was already a fresh copy. Wrote a test case and fixed by making |
| 97 | builtin_apply copy. Also broken in tinyscheme. Affects what is probably the |
| 98 | simplest way to copy a list, (apply list some-list). |
| 99 | |
| 100 | Major refactoring: compiler, variable references... |
| 101 | |
| 102 | October 2017: |
| 103 | |
| 104 | 9. Register effect bugs in assoc and member builtins (see tests-misc.scm). |
| 105 | |
| 106 | 10. Miscompilation of "delay" (see tests-misc.scm). |
| 107 | |
| 108 | 11. The compiler was not immune to redefinition of builtins. (REPL too?) |
| 109 | Implemented immutable environments with a separate copy for the interaction |
| 110 | environment. Immutable string/list/vector constants still need doing. |
| 111 | |
| 112 | Implemented piped subprocess extension |
| 113 | |
| 114 | 12. Realized input_port()/output_port() could leak the file object and |
| 115 | descriptor if they hit an out of memory error. |
| 116 | |
| 117 | November 2017: fuzz testing |
| 118 | |
| 119 | 13. Infinite loop in the lexer for EOF in a comment. |
| 120 | |
| 121 | Implemented CLI options, memory stats, floor/ceiling/truncate/round/port? |
| 122 | |
| 123 | January 2018: |
| 124 | |
| 125 | 14. Heap discipline broken in arithmetic builtins on flonums by way of "fold" |
| 126 | helper. |
| 127 | |
| 128 | April 2018: |
| 129 | |
| 130 | Partial implementation of the long-missing number->string and string->number |
| 131 | builtins, enabling removal of a bunch of printfs. |
| 132 | |
| 133 | 15. Discovered by inspection that the open_file helper was unsafely storing a |
| 134 | port in a local variable across a call to string_append_null. Made it pass the |
| 135 | port by register instead and refactored a bit. |
| 136 | |
| 137 | 16. Realized that the changes implemented for #12 can't actually solve the problem; |
| 138 | file objects and descriptors still leak if a subsequent unrelated call gets the |
| 139 | memory error. Backed them out in order to simplify port construction. |
| 140 | |
| 141 | 17. builtin_open_subprocess was a mess all around. Changed the interface, |
| 142 | improved comments, simplified the temporary argv construction (resolving a |
| 143 | strict aliasing warning), and tightened syscall error handling. As above, |
| 144 | realized that leaking resources due to allocation failures is not a problem |
| 145 | that can be solved within the scope of this function (so e.g. it's fine to |
| 146 | allocate the return value list after the child is spawned). |
| 147 | |
| 148 | 18. REPL now accepts multi-valued returns, printing them one per line. |
| 149 | |
| 150 | 19. Extensions are now available in an immutable (gales-scheme-environment), |
| 151 | analogous to (scheme-report-environment 5), in addition to the interaction |
| 152 | environment. This allows for hygienic packages making use of the extensions |
| 153 | without dependency injection. Probably useless flush-input-port removed. Some |
| 154 | toplevel error handler setup inconsistencies fixed, but the error plumbing |
| 155 | needs more scrutiny. |
| 156 | |
| 157 | 20. Implemented double-width multiply functions, signed and unsigned, in |
| 158 | portable C and x86_64 assembly, plus range checking and float conversion |
| 159 | helpers. Used these to fix long-standing overflows in * and expt builtins. |
| 160 | Implemented extensions for wrapped, carried and bitwise arithmetic on fixnums. |
| 161 | Factored tagging macros out of machine-specific #if-blocks and tightened their |
| 162 | types by casting. Need to finish correctness proof for signed wide multiply. |
| 163 | |
| 164 | May 2018: |
| 165 | |
| 166 | 21. Fixed self-initialization bug in macro expansion in fx-/wrap builtin (and |
| 167 | added -Winit-self to gcc flags). Added fx-/carry-unsigned and fxmaj builtins. |
| 168 | Made some internal functions static. Added startup assertion to enforce the |
| 169 | assumption that unsigned long matches the value / pointer size. Minor tweaks to |
| 170 | build in C99 pedantic mode; would like to move toward C89. |
| 171 | |
| 172 | 22. Implemented if-exists extension for open-output-file. Switched to "let" |
| 173 | forms in toplevel. Made static symbol variable setup/usage more consistent. |
| 174 | |
| 175 | 23. Core: |
| 176 | - Catch I/O errors in flush_port, read_char and peek_char, simplifying spec for |
| 177 | flush-output-port builtin. |
| 178 | - Light comment editing. Remove specification comments for nonstandard |
| 179 | builtins, which now have their own extensions document. |
| 180 | - Make eqv? and equal? do the simpler checks first, short-circuiting deep |
| 181 | inspection where possible. |
| 182 | - Rename quit builtin to exit. |
| 183 | - Add a GC thrash factor (knob not exposed yet) to hasten OOM detection when GC |
| 184 | reclaims less than heap_size/factor space. |
| 185 | - Wipe registers on OOM detection (previously just the stack was dropped in |
| 186 | sc_error1, with reason unclear). |
| 187 | - Rename fallback error handlers to fatal/fatal1 for clarity. |
| 188 | |
| 189 | Core/compiler/toplevel: |
| 190 | - Implement "error" as a normal builtin, parallel to sc_error1, making |
| 191 | set-error-handler! work for Scheme-signalled errors too and removing the |
| 192 | compiler's special error procedure with its null-terminator wrangling. |
| 193 | - Remove syn-env from the special compiler environment as it can be handled |
| 194 | purely from Scheme. |
| 195 | - Merge the compiler environment (now just one binding) into the privileged |
| 196 | toplevel environment. |
| 197 | |
| 198 | Toplevel: |
| 199 | - Extend call-with-output-file and with-output-to-file to pass through the |
| 200 | if-exists option. |
| 201 | - Make with-input-from-file and with-output-to-file restore the current ports |
| 202 | on escape or reentry, not required by spec but clearly the right thing. |
| 203 | (Using a stub dynamic-wind, so won't actually work yet.) |
| 204 | - Fix all four of those to work with multi-valued returns. |
| 205 | - Make REPL error handler support multiple detail arguments (though the other |
| 206 | components don't yet). |
| 207 | |
| 208 | 24. Added sync and data-sync options to flush-output-port extension. |
| 209 | Refactoring of port argument wranglers. |
| 210 | |
| 211 | 25. Big move of builtins to library. Startup time and performance of the |
| 212 | builtins in question is degraded, but much hazardous and hard-to-verify |
| 213 | microcode is eliminated. |
| 214 | |
| 215 | Removed compiler dependence on builtin forms: |
| 216 | and |
| 217 | or |
| 218 | |
| 219 | Removed compiler dependence on builtin procedures: |
| 220 | append |
| 221 | apply |
| 222 | assq |
| 223 | for-each |
| 224 | list |
| 225 | list-tail |
| 226 | map |
| 227 | memq |
| 228 | vector->list |
| 229 | zero? |
| 230 | |
| 231 | Compiler now depends on some extensions and uses eq? for fixnum comparison, in |
| 232 | hopes of reducing performance impact on replaced builtins. |
| 233 | |
| 234 | Merged the syntax library into the toplevel, removing some startup complexity |
| 235 | and allowing the toplevel to use library syntax. |
| 236 | |
| 237 | Fixed some edge cases of the "case" macro. |
| 238 | |
| 239 | The "load" builtin, by way of exec-from-port, now reads the whole file before |
| 240 | evaluating the forms. |
| 241 | |
| 242 | Forms moved from core to syntax library: |
| 243 | and |
| 244 | or |
| 245 | |
| 246 | Builtins moved to library: |
| 247 | list |
| 248 | append |
| 249 | reverse |
| 250 | list-tail |
| 251 | list-ref |
| 252 | memq |
| 253 | memv |
| 254 | member |
| 255 | assq |
| 256 | assv |
| 257 | assoc |
| 258 | vector->list |
| 259 | apply |
| 260 | map |
| 261 | for-each |
| 262 | |
| 263 | New privileged builtins: |
| 264 | apply/unchecked |
| 265 | car/unchecked |
| 266 | cdr/unchecked |
| 267 | set-cdr/unchecked! |
| 268 | vector-ref/unchecked |
| 269 | fx+/unchecked |
| 270 | |
| 271 | Wrote some tests for the new library procedures. Accelerated negative tests by |
| 272 | replacing bash/sed gunk with a custom error handler to run all in one process. |
| 273 | Moved the currently failing immutability test to the negatives and expanded. |
| 274 | |
| 275 | June 2018: |
| 276 | |
| 277 | 26. Make "string-append" run in linear rather than quadratic time, fixing a |
| 278 | FIXME. Make toplevel "apply" and "append" list copying helpers work tail |
| 279 | recursively, by mutation in one pass as in the original C, reducing memory |
| 280 | overhead. |
| 281 | |
| 282 | 27. Implemented dynamic-wind by adding a "spool" register and a new field in |
| 283 | continuation objects to capture it, the necessary additions to the APPLY |
| 284 | subroutine (now separated into APPLY_CONTINUATION), and a lowest common |
| 285 | ancestor helper function. |
| 286 | |
| 287 | Substantially refactored error handling in both core and toplevel to: |
| 288 | * work with dynamic winding while avoiding infinite loops; |
| 289 | * avoid some unnecessary C to Scheme string conversions; |
| 290 | * remove string_append helper, a poor interface (per change 26); |
| 291 | * support multiple detail arguments to ERROR, per SRFI-23; |
| 292 | * be generally clearer, I think. |
| 293 | |
| 294 | Similarly, the EXIT extension is now implemented using a continuation that |
| 295 | returns the given code from the toplevel (and sc_toplevel now returns the code |
| 296 | to main). This both makes it support unwinding and removes the need for a |
| 297 | builtin exposing exit(). (At some point it might be nice to eliminate remaining |
| 298 | uses of exit(), but so far it's still needed as an error handling fallback.) |
| 299 | |
| 300 | Builtins moved to library: |
| 301 | string |
| 302 | string->list |
| 303 | vector |
| 304 | set-error-handler! |
| 305 | exit |
| 306 | |
| 307 | New privileged builtins: |
| 308 | push-winding! |
| 309 | string-ref/unchecked |
| 310 | set-error-continuation! |
| 311 | |
| 312 | Extension added to library: |
| 313 | error |
| 314 | |
| 315 | builtin_error still exists, but only for reporting startup errors before it's |
| 316 | replaced by the toplevel. |
| 317 | |
| 318 | Fixed a few cases where pointers were being followed into the heap without |
| 319 | untagging. This had gone unnoticed because the three tag bits happen to |
| 320 | overflow in the shift for array indexing on 64-bit. |
| 321 | |
| 322 | Some minor clarity and error context improvements to library procedures. |
| 323 | |
| 324 | Renamed list_length/unsafe_list_length, to avoid giving the impression that |
| 325 | safety can be assumed for these internal helper functions. |
| 326 | |
| 327 | Some C89ist shuffling of declarations. |
| 328 | |
| 329 | Optimized builtin_cons: reuse pair from arg list as it's freshly allocated. |
| 330 | |
| 331 | sc_toplevel now returns the integer exit code, |
| 332 | |
| 333 | TODO: think thorough / test what happens on error or other escape in |
| 334 | unwinding/rewinding, explicitly unspecified by R5RS. |
| 335 | |
| 336 | 28. Minor optimization: compile quoted empty lists and quoted vectors to the |
| 337 | values themselves, as they're not valid expressions and thus can be made |
| 338 | self-evaluating simply by removing the check. Deprioritize "quote" in the |
| 339 | builtin syntax dispatch (based on pure guesswork as to frequency). 1% speedup |
| 340 | observed in some existing tests, 5% in a contrived one. |
| 341 | |
| 342 | Condense some redundant register aliases (R_OPERATOR->R_EXPR; |
| 343 | R_BODY->R_OPERANDS), avoiding some pedantic self-assignment. |
| 344 | |
| 345 | 29. Fixed a subtle semantics bug in FORCE, and wrote test: a promise's code may |
| 346 | be entered more than once if it forces itself recursively; if results differ, |
| 347 | the first must not be clobbered. Found by re-reading rationale in R5RS. |
| 348 | |
| 349 | In the same spirit of "mutability is not your friend", reverted the |
| 350 | builtin_cons optimization from #27. While safe, arglist mutation is premature |
| 351 | at this point; for example it may hinder debugging, which ought to be a |
| 352 | strength of an interpreter. If I do decide to allow it, there's a bunch of |
| 353 | other builtins that could do it too. But a better optimization might be to |
| 354 | avoid building arglists in the first place for fixed-arity builtins. |
| 355 | |
| 356 | 30. New builtins, with tests: |
| 357 | immutable? |
| 358 | cons/immutable |
| 359 | string-copy/immutable |
| 360 | vector-copy/immutable |
| 361 | |
| 362 | Slightly simplified some related code. |
| 363 | |
| 364 | In previous immutability work, rather than expose new primitives I had the |
| 365 | reader construct strings and vectors as immutable. This was neither correct nor |
| 366 | useful. Literal constants are now properly petrified by the compiler, and with |
| 367 | that, all my tests are finally passing. |
| 368 | |
| 369 | 31. Mega-IO-refactor, replacing stdio with Scheme-managed buffering, and |
| 370 | implementing sockets extension. |
| 371 | |
| 372 | Port-based write functions now all use R_PORT rather than an argument and |
| 373 | signal errors rather than returning status. This tidies up the printer quite a |
| 374 | bit. |
| 375 | |
| 376 | Checking for open port on each read/write no longer needed - the kernel's FD |
| 377 | validity check is enough. |
| 378 | - Added bonus: (close-output-port (current-output-port)) at REPL was a segfault |
| 379 | as the check had been missed in the fallback error handler. Now "FATAL: Bad |
| 380 | file descriptor". |
| 381 | - Should probably prevent closing stdin/out though. (close-input-port |
| 382 | (current-input-port)) at REPL is an infinite loop. |
| 383 | |
| 384 | Replaced posix_spawn* with vfork, simplifying builtin_open_subprocess, possibly |
| 385 | making it fast on more platforms, and removing another likely malloc user. |
| 386 | |
| 387 | New privileged builtins operating at the file descriptor level: |
| 388 | inet-stream-socket |
| 389 | inet-dgram-socket |
| 390 | unix-stream-socket |
| 391 | unix-dgram-socket |
| 392 | socket-ports |
| 393 | getsockname |
| 394 | getpeername |
| 395 | connect-inet |
| 396 | connect-unix |
| 397 | listen |
| 398 | accept |
| 399 | close |
| 400 | |
| 401 | New library procedures: |
| 402 | open-tcp-connection |
| 403 | open-tcp-listener |
| 404 | open-udp-socket |
| 405 | open-unix-connection |
| 406 | open-unix-listener |
| 407 | open-unix-datagram-socket |
| 408 | call-with-tcp-connection |
| 409 | call-with-unix-connection |
| 410 | sequential-tcp-server |
| 411 | sequential-unix-server |
| 412 | |
| 413 | 32. Fix assertion violation (buffer overflow) introduced in #31, when writing |
| 414 | to a closed port, catching the error, then writing again. |
| 415 | |
| 416 | Relatedly, any other write errors on the underlying FD will now close the port. |
| 417 | (Rationale: if recovery were possible, the underlying system should already be |
| 418 | doing it, so write errors are generally permanent; even if not, they can be |
| 419 | asynchronous so no way to know how much was successfully written.) |
| 420 | |
| 421 | REPL error handler changed back to restarting using a captured continuation. (I |
| 422 | think I had removed this deliberately in #27 in the spirit of simplifying, but |
| 423 | the result was the REPL exiting with error status on a normal ^D termination if |
| 424 | an error had ever been caught, because error handlers aren't really supposed to |
| 425 | return.) |
| 426 | |
| 427 | Ugly (define-r5rs 'foo (lambda ...)) forms changed to internal definitions |
| 428 | followed by bulk registration, in the same manner as the extensions in #31. |
| 429 | |
| 430 | 33. New builtin: char-ready?, made possible by direct access to input buffers. |
| 431 | Stream sockets are made nonblocking to work around poll/select defects, but |
| 432 | undetected blocking is still possible, e.g. reading from disk or if standard |
| 433 | input is an externally provided socket. |
| 434 | |
| 435 | Builtin moved to library and tests added: equal? |
| 436 | |
| 437 | 34. Libc in-sourcing: |
| 438 | - Removed stdio from main.c and indirect use via assert |
| 439 | - Removed last direct uses of malloc by replacing GC root linked list with an |
| 440 | array and heap creation with mmap, also enabling the hugepages option |
| 441 | - Removed last indirect use of malloc via atexit |
| 442 | - Replaced stdlib.h/string.h with individual declarations |
| 443 | |
| 444 | Made write_err handle EINTR/EAGAIN/EWOULDBLOCK, sharing with flush_output_port. |
| 445 | |
| 446 | July 2018: |
| 447 | |
| 448 | 35. Start of library-level bignum support, including addition, subtraction, |
| 449 | Comba multiplication, and automatic fixnum promotion and demotion. Using |
| 450 | non-opaque values until the proper privileged interface is settled. |
| 451 | |
| 452 | It looks like the reader and printer will have to be done in Scheme now. |
| 453 | (They'd have to make non-tail calls back into Scheme to do base conversion, and |
| 454 | such C-level recursion is verboten.) So much the better for sanity and future |
| 455 | rewrites of the core though. A pared-down bootstrap reader will be preserved. |
| 456 | |
| 457 | Core: |
| 458 | - More assertions |
| 459 | - Flush ports on abort (with re-entry check, as flushing itself has assertions) |
| 460 | - Bignum awareness in number predicates |
| 461 | - Start replacing car(cdr(x)) pattern with cadr macro |
| 462 | |
| 463 | Builtins moved to library: + - * |
| 464 | |
| 465 | New builtin: fx</unsigned |
| 466 | |
| 467 | New privileged builtins: |
| 468 | fx-/unchecked |
| 469 | fx=/unchecked |
| 470 | fxneg/unchecked |
| 471 | fxnegative/unchecked? |
| 472 | bignum? |
| 473 | flonum? |
| 474 | flonum/unchecked |
| 475 | floneg/unchecked |
| 476 | flo+/unchecked |
| 477 | flo-/unchecked |
| 478 | flo*/unchecked |
| 479 | |
| 480 | Renamed builtin: fx-/carry-unsigned to fx-/borrow-unsigned. (Makes more sense |
| 481 | to me as the overflow output is 1, not -1.) |
| 482 | |
| 483 | Syntax library: |
| 484 | - SRFI-8 RECEIVE syntax to simplify call-with-values. (LET-VALUES is nicer but |
| 485 | ugly to implement.) Ideally this wouldn't be visible in the R5RS environment. |
| 486 | - Fix quirk where COND with no successful tests returned #f rather than the () |
| 487 | I'm generally using for "unspecified". This also worked around the compiler |
| 488 | bug that caused it to reject a literal #f as the final test. |
| 489 | |
| 490 | 36. Serious bugfix: missed making the spool register a GC root in #27. |
| 491 | |
| 492 | Minor bugfixes: |
| 493 | - Proper PEEK-CHAR semantics for devices with transient end-of-file e.g. tty. |
| 494 | - Detect excess arguments in the helper macro for unary character functions. |
| 495 | |
| 496 | New builtin: read-token |
| 497 | |
| 498 | New privileged builtin: reverse-list->vector/unchecked |
| 499 | |
| 500 | Builtins moved to library: read string->number |
| 501 | |
| 502 | The full range of numeric syntax is now supported except for rational and |
| 503 | complex: alternate radix (2/8/10/16), radix and exactness prefixes, exponents |
| 504 | and sharped-out digits (though significant figures aren't actually tracked). |
| 505 | Bignums come "for free", though base conversion is the naive quadratic |
| 506 | algorithm. |
| 507 | |
| 508 | Simplified the (now) bootstrap number decoder by removing float syntax and |
| 509 | promotions. |
| 510 | |
| 511 | Internal tweaks to the lexer and (now) bootstrap reader to support read-token. |
| 512 | Also loosened the overly anal interpretation of allowed whitespace. |
| 513 | |
| 514 | Started downcasing labels (all-caps doesn't really serve a purpose and risks |
| 515 | interference from libc extensions). |
| 516 | |
| 517 | 37. Bignum division, conversion to flonum and output. |
| 518 | |
| 519 | Bugfix: bignum negation failed to demote in the case of *LEAST-FIXNUM*. |
| 520 | |
| 521 | Minor optimization: avoid some unnecessary untagging in fixnum ops. |
| 522 | |
| 523 | Builtins moved to library: |
| 524 | zero? |
| 525 | positive? |
| 526 | negative? |
| 527 | abs |
| 528 | quotient |
| 529 | remainder |
| 530 | modulo |
| 531 | exact->inexact |
| 532 | number->string |
| 533 | write |
| 534 | display |
| 535 | newline |
| 536 | |
| 537 | In addition to bignum support, quotient/remainder/modulo now properly maintain |
| 538 | inexactness for flonum inputs, though can produce inf/nan. |
| 539 | |
| 540 | In the Scheme rewrite of the printer, I realized the special save/restore of |
| 541 | current ports for error recovery is unnecessary now that dynamic-wind is taking |
| 542 | care of it (they can't be set directly by user code). |
| 543 | |
| 544 | New library procedures: gcd lcm |
| 545 | |
| 546 | New builtins: fx= fx< fx<= fx<=/unsigned fxlength/unsigned |
| 547 | |
| 548 | Renamed builtin: fxshift-unsigned to fxshift/unsigned |
| 549 | |
| 550 | New privileged builtins: |
| 551 | fxdiv/unchecked |
| 552 | fxdiv/ext/unchecked |
| 553 | fixnum->dec/unchecked |
| 554 | fixnum->hex/unchecked |
| 555 | fixnum->oct/unchecked |
| 556 | fixnum->bin/unchecked |
| 557 | flonum->dec/unchecked |
| 558 | flonum/unsigned/unchecked |
| 559 | flodiv/unchecked |
| 560 | floquotient/unchecked |
| 561 | floremainder/unchecked |
| 562 | builtin? |
| 563 | builtin-name |
| 564 | promise? |
| 565 | continuation? |
| 566 | |
| 567 | 38. Full bignum integration. |
| 568 | |
| 569 | Serious bugfix: (QUOTIENT *LEAST-FIXNUM* -1) => *LEAST-FIXNUM* |
| 570 | (On the first pass for #37 it didn't occur to me that signed fixnum quotient |
| 571 | had an edge case besides zero divisor. The equivalent problem existed in the |
| 572 | prior C-only implementation, or possibly worse as C leaves negative division |
| 573 | implementation-defined.) |
| 574 | |
| 575 | Builtins moved to library: |
| 576 | eqv? |
| 577 | = < > <= >= |
| 578 | odd? even? |
| 579 | max min |
| 580 | / |
| 581 | floor ceiling truncate round |
| 582 | exp log |
| 583 | sin cos tan |
| 584 | asin acos atan |
| 585 | sqrt |
| 586 | expt |
| 587 | inexact->exact |
| 588 | |
| 589 | Bignums now opaque and supported by all generic operators. Unlike the fixnum |
| 590 | and flonum ops, which could also exist as unprivileged extensions, the bignum |
| 591 | primitives are implicitly unchecked. In addition to bignum support, the |
| 592 | irrational functions now handle some special cases for exactness e.g. zero (as |
| 593 | encouraged but not required). |
| 594 | |
| 595 | New privileged builtins: |
| 596 | toplevel-environment |
| 597 | set-car/unchecked! |
| 598 | fx</unchecked |
| 599 | fx<=/unchecked |
| 600 | fixnum->bin/unsigned/unchecked |
| 601 | flo=/unchecked |
| 602 | flo</unchecked |
| 603 | flo<=/unchecked |
| 604 | fraction/exponent/unchecked |
| 605 | load-exponent/unchecked |
| 606 | inf/unchecked? |
| 607 | flonum->fixnum/unchecked |
| 608 | floor/unchecked |
| 609 | ceiling/unchecked |
| 610 | truncate/unchecked |
| 611 | round/unchecked |
| 612 | exp/unchecked |
| 613 | log/unchecked |
| 614 | sin/unchecked |
| 615 | cos/unchecked |
| 616 | tan/unchecked |
| 617 | asin/unchecked |
| 618 | acos/unchecked |
| 619 | atan/unchecked |
| 620 | atan2/unchecked |
| 621 | sqrt/unchecked |
| 622 | make-bignum |
| 623 | bignum? |
| 624 | bignum-negative? |
| 625 | bignum-set-negative! |
| 626 | bignum-ref |
| 627 | bignum-set! |
| 628 | bignum-length |
| 629 | bignum |
| 630 | bignum/unsigned |
| 631 | bignum2 |
| 632 | bignum-truncate! |
| 633 | |
| 634 | Renamed privileged builtins: |
| 635 | fxdiv/unchecked -> fxdiv/unsigned/unchecked |
| 636 | fxdiv/ext/unchecked -> fxdiv/ext/unsigned/unchecked |
| 637 | |
| 638 | (They were already unsigned; I think I'd been trying to avoid the crazy long |
| 639 | names, but it was a hazard.) |
| 640 | |
| 641 | Core: |
| 642 | - Grabbed the last extended type slot to fit the bignum sign bit |
| 643 | - Type assertions added to fixnum_val, new counterpart unsigned_fixnum_val, and |
| 644 | flonum_val, expanding coverage while reducing boilerplate |
| 645 | - Bounds check assertion added to vector_ref and vector_set |
| 646 | - With flonum coercion headaches out of core, finally replaced safe_fixnum_val |
| 647 | and exact_fixnum_val with something simple and correct |
| 648 | - is_number optimized based on extended number type tag values being contiguous |
| 649 | - is_integer made possibly more standard conformant by using ROUND (nearbyint) |
| 650 | instead of FLOOR for flonums (don't see how this could make a difference but |
| 651 | neither am I sure it can't) |
| 652 | - Made shallow_print report particular environment specifiers, the internal |
| 653 | specials, immutability, and vector lengths |
| 654 | - make_str and make_vector size checks simplified based on the unsigned trick |
| 655 | |
| 656 | Compiler: |
| 657 | - Removed fairly useless vector support from macro language |
| 658 | - Fixed old bug: improper rejection of macros expanding to #f |
| 659 | - Removed trivial dependency on generic < |
| 660 | - Switched to storing error context as symbol, avoiding SYMBOL->STRING calls in |
| 661 | the normal case |
| 662 | |
| 663 | Toplevel: |
| 664 | - Pretty "evaluates to" symbol prefixing REPL results |
| 665 | - Tracing for library procedure calls, allowing removal of the growing sprawl |
| 666 | of ad-hoc error context reporting |
| 667 | - Filled in stubs for bignum to hex, octal and binary conversion |
| 668 | - Handled bignum status in EXIT by remaindering there rather than in core |
| 669 | |
| 670 | asm-generic: |
| 671 | - Replaced "seems right but can't prove" implementation of signed double-width |
| 672 | multiply with ugly but more obvious method |
| 673 | |
| 674 | August 2018: |
| 675 | |
| 676 | 39. |
| 677 | Serious bugfix: plug leaks of uncompiled subtrees in named LET and DO, while |
| 678 | simplifying, by feeding the full expansions back through compilation as is done |
| 679 | for ordinary macros. (Less seriously, the same for lambda annotation in plain |
| 680 | LET, at worst a performance issue.) |
| 681 | |
| 682 | Core: |
| 683 | - Add is_port assertions to R_PORT-based I/O routines |
| 684 | - Remove unnecessary variable pre-check in SET, allowing resolve_variable_ref |
| 685 | to operate on unresolved refs only, saving a few branches |
| 686 | - Remove fairly pointless and expensive is_list assertion in EVAL |
| 687 | - Remove redundant is_fixnum assertion from builtin_make_bignum |
| 688 | |
| 689 | Compiler: |
| 690 | - More consistently unquote self-evaluating objects |
| 691 | - Reject non-readable objects (possible through EVAL) |
| 692 | |
| 693 | Syntax library: |
| 694 | - OR: optimize by avoiding unnecessary temporary binding (multiple evaluation |
| 695 | is safe for non-list expressions as they're all side-effect free and O(1)) |
| 696 | - Explain how the hardcoded temporary names are safe |
| 697 | - COND: simplify by handling the temporary binding case with OR |
| 698 | - CASE: optimize by using EQV? instead of MEMV for the single-datum case |
| 699 | |
| 700 | 40. |
| 701 | Core/compiler: |
| 702 | - Prevent integer overflows by limiting parameter list length (number of |
| 703 | bindings per lexical frame) |
| 704 | - Save a word in the procedure object and a pair in the annotated lambda form |
| 705 | by encoding variadic status in the sign of the arity |
| 706 | |
| 707 | Core: |
| 708 | - Avoid possibly undefined overflow behavior in untag_signed by doing the left |
| 709 | shift as unsigned |
| 710 | - Note a hazard in add_tag/ext_add_tag and audit |
| 711 | |
| 712 | Compiler: |
| 713 | - Slight simplifications/optimizations: filterize CHECK-IDENT; avoid repeated |
| 714 | SYMBOL->STRING in duplicate search |
| 715 | |
| 716 | 40.maint1. |
| 717 | |
| 718 | Starting a maintenance branch as the crucial variable lookup optimization has |
| 719 | not yet made it back after the compiler overhaul of revision 41. |
| 720 | |
| 721 | Core: |
| 722 | - Remove assertions in stack ops, redundant since adding type assertions to |
| 723 | car/cdr long ago |
| 724 | - Remove invalid assertion added in r39: input_port_ready doesn't use R_PORT |
| 725 | - Remove mostly redundant is_list assertions in the full environment lookups |
| 726 | |
| 727 | March 2020: |
| 728 | |
| 729 | 40.maint2. |
| 730 | |
| 731 | Overall: |
| 732 | - Restructure tree for slashpackage |
| 733 | - Trim Makefile |
| 734 | - Write install + check scripts and README |
| 735 | |
| 736 | Main (backports): |
| 737 | - Make default heap size build-time tunable (isolating magic numbers) |
| 738 | - Report build configuration in help text |
| 739 | - Remove "strtol" stdlib dependency |
| 740 | |
| 741 | 40.maint3. |
| 742 | |
| 743 | Bulk reindent & reflow. |
| 744 | - Tabs: good |
| 745 | - Non-meaningful newlines in human text: bad |
| 746 | - "if" as a special "lisp word" producing less indentation: bad |
| 747 | - Code width: mostly sticking to 80 columns |