Fixpoint

2019-12-11

Uruguay parte 2: llegada y primeras vistas

Filed under: Historia, Politikos, Vita — Jacob Welsh @ 20:12

Continued from Parte 1.

My text having overtaken the start of photography, I'll have to backtrack a bit to Montevideo's Aeropuerto Internactional de Carrasco (MVD) which was looking quite shiny and new. Bag claim (evidently I misremembered: there were three on the international side, though just the one active):

uy-2

Aduanas (customs). That bienes de ingreso/egreso temporal would seem vague enough to cover just about anything if they felt like it; fortunately they didn't give a second glance (perhaps even first glance) to my scandalous screwdriver and packing materials.

uy-1

Free at last, but not quite home.

uy-5

I would have picked up a local SIM but the booth was closed for the night. It turned out my Panama SIM worked on roaming, at least briefly, which it hadn't in the US.

uy-3

The bit of the world that is me thanks Uruguay for the welcome.

MUNDO, BIENVENIDO A URUGUAY

In contrast to Panama, there was no crowd of taxi syndicate reps soliciting eagerly. Instead it's an orderly racket; you go to the taxi counter and arrange a ride with prepayment and receipt. Having been warned the cab would be around US $55, I held out for the $13 shuttle bus, taking the wait time to replace that stolen sunscreen, collect my thoughts and decompress a bit. I found myself tired but alert and relieved.

The only exterior shot I managed of the airport, so it'll have to do:

uy-6

Some Himpton by Halton thing near the airport with well-lit street:

uy-7

First shuttle stop was at the Motel Bahamas:

uy-8

A pleasant nighttime drive down the coast and another one or two stops later and I'd made it to my destination in the relatively nice Pocitos neighborhood.

uy-10

uy-9

The buildings here cap out around 10-12 stories due to zoning. Most are mixed-use, with shops at ground level and apartments above. My first impression of the area compared to most of Panama City was of something older (turns out many buildings date to the 1930's if I recall), more stable (as opposed to wreckage and new construction everywhere), cleaner, and far more pedestrian friendly (wide and not entirely treacherous sidewalks). Aaron pointed out that this does not apply to the whole city, with outlying neighborhoods ranging from more typically LatAm to outright favela (though these not walled off as in Brazil).

uy-11

Right around the block was an ANCAP gas station with rare 24-hour convenience store and deli, which served me for breakfasts, rather dreadful espresso (they couldn't believe I didn't want sugar, which probably says it all), and a printed map so as to navigate free of any "mobile device" nonsense.

First daytime views of the coastal Rambla, supporting vehicle, bike and pedestrian traffic and beach access, as I made my way to meet my host.

uy-12

uy-13

Oh yes, the street signs serve advertising; it does seem to help keep them in good shape.

uy-14

One of the larger mini-parks opposite the beach, near the Avenida Brasil.

uy-15

Battery scooters for hire.

uy-16

"Por la vida y la convivencia" : La Policia seem to like their mottos...

uy-17

There's a lazy tourism option.

uy-18

I have no idea. It didn't seem animate.

uy-19

"Orgullosamente blanco" - "proudly white" - referring, I gather, to the party colors of the recently victorious Partido Nacional rather than something racial.

uy-20

Either Ave. Brasil or Espana, two thoroughfares that converge at the coast.

uy-21

Corner florists seem to be thriving...

uy-22

Corner locksmiths not so much.

uy-23

Heading inland a bit; some gym/yoga place.

uy-24

uy-25

Apparently they don't need no education at the Center of Foreign Tongues. Aaron tells me the buildings generally don't get repainted much because maintenance work is taxed the same as new construction.

uy-26

Perhaps this would have been the spot for a better coffee.

uy-27

uy-28

There's minimal piped natural gas infrastructure (as in Panama, though there it's often provided building-wide and refueled by tanker trucks).

uy-29

Pizzeria Trouville

Another florist, and some of the typical sycamores lining the streets.

uy-31

To be continued with rare electronics and proper tourism.

2019-12-10

Una visita a la Republica Oriental del Uruguay, parte 1

Filed under: Politikos, Vita — Jacob Welsh @ 18:22

Having bought some of the remains of the historic but sadly liquidated Bitcoin firms No Such lAbs and Pizarro ISP, and with expected overseas shipping costs being comparable to a personal courier run, I seized the opportunity for some travel and networking. It's been a success on all three fronts: retrieving the gear, getting a taste of Montevideo, and meeting and spending some quality time with Aaron Rogier aka BingoBoingo, whom I'd previously known mainly as the humorously grandiose voice of Qntra and a thoughtful contributor to IRC discussions; I found him to demonstrate the same insight in person and be quite likable besides. I made it a three-night stay to allow one full day for the hauling and packing and one for tourism.(i)

My biggest mess-up of the trip as I see it was not allowing enough time for my initial departure from the "Hub of the Americas", Panama's recently expanded Tocumen International Airport - for which I'm starting to develop a hearty loathing - or the time to get there, my previous departures here having been either in the wee hours or from locations with better toll road access. I got stuck in the check-in cattle queue for the better part of an hour.(ii) By the time my turn came, I was informed that due to late check-in my bags would be subject to "voluntary separation" and might end up on the next flight. Since apparently I couldn't find out whether they made it until arrival, I worried and contemplated my options on the flight. At security, while not subjected to the gate-side mandatory gropings reserved for the US-bound, there were still US-inspired theatrics like shoe removal and inspecting my carry-on for liquids, confiscating my over-100ml sunscreen. Serves me right for being such a terrorist, huh.

Things went much smoother from there; immigration in Montevideo was a breeze at least for chip-enabled passport holders, there were no kilometers to walk to the airport's one baggage claim, and my bags had made it just fine. Having been warned about the pricey airport taxi service, I elected to wait for a shuttle, which departed once the next flight had dumped enough passengers to form a group. On exiting the airport (around 2am local time) I was welcomed by the delightfully cool, spring-like air: always a nice thing after months in the tropics, though my skin and nose didn't adjust to the dryness too well.

All the travel intel Aaron had given that I had chance to verify proved accurate, and the Punta Trouville hotel he recommended was the perfect fit for my needs: budget but clean, functional, well located and with 24-hour service. Power outlets and money proved easier than anticipated. The hotel had multi-format outlets; it's just as well I came prepared with adapters, as Aaron said those can be flaky, though they worked for me. While there are cambios all over for changing currency with around 4% spread, I never ended up needing one as the airport transit and the merchants I tried were all equipped and even glad to take my specie (well, USD) and give change in pesos Uruguashos; the local currency sees the sort of inflation that gets automatically priced into yearly contracts.

To be continued (and with photos).

  1. Not ideal for really getting to know a place, but I already had a longer holiday coming up and lots to get done before it. [^]
  2. The "web check-in" line turned out to move faster; I can't see any good reason as it doesn't save much time at the counter: you still need to get docs checked, bags weighed and tagged, and any overage paid. The main reason as far as I could tell was simply that they'd allocated more agents there and didn't rebalance until the line was entirely exhausted. [^]

2019-12-05

Basic getrawtransaction patch proposal for bitcoind

Filed under: Bitcoin, Software — Jacob Welsh @ 17:35

I present a bitcoind patch, of my own authorship, for consideration: a basic but functional getrawtransaction RPC command. I expect the need for it is clear: if it's your node then you shouldn't accept any sort of "I'm sorry Dave, I'm afraid I can't do that" especially regarding the data it already has handy.

To speak plainly about some deficits, this time not my own: the Real Bitcoin project presently rests on some shaky foundations. Despite apparently intricate efforts, a Keccak based vtree has seen little progress, in that the original patch signers besides mod6 have not signed on to the regrinds of their work, and some recommended patches have not been reground at all, updated for the very necessary manifest spec, or included in the mainline collection. Further, the sorry state of the inherited code such as magic numbers everywhere has seen little improvement. Perhaps I will have to take up some of these burdens in time; for now I'll leave it as an entreaty to the elder hands to please find a way to make it happen!

The patch

As in the original introduced somewhere around PRB 0.7.0, this command takes a transaction ID (256 bits of hex), searches first the node's own memory pool then the confirmed transaction database (blkindex.dat) and returns a hex dump of the encoded transaction. Unlike the original it does not support a "verbose" option to give a JSON representation of the data. This task seems to me better suited to an external tool, but I could see including it here if the implementation is concise and obviously correct.

Backporting the original was not possible due to the many intervening changes, though I did consult it to confirm I hadn't missed anything important and matched its numeric error code.

Based on the overall idea in the V version control system of building yggdrasil, I'm breaking from one of the project's prior naming conventions by including "bitcoin" but not the author in the patch name; the author is still recorded in the enclosed manifest file. Due to the problems noted earlier with the prior patch tree it's also not a proper vpatch yet.

Download: bitcoin_getrawtransaction.draft.patch.

To try this you will need to:

  1. Perform a press to asciilifeform_whogaveblox.vpatch;
  2. Manually apply mod6_phexdigit_fix.vpatch (which could be missed otherwise due to lacking a manifest entry);
  3. Manually apply the patch in question.

In detail

I added a manifest entry for the phexdigit fix, to make its inclusion explicit:

--- a/bitcoin/manifest.txt
+++ b/bitcoin/manifest.txt
@@ -28,3 +28,5 @@
 542413 asciilifeform_aggressive_pushgetblocks asciilifeform Issue PushGetBlocks command to any peer that issues 'version' command
 542413 mod6_excise_hash_truncation mod6 Regrind of ben_vulpes original; removes truncation of hashes printed to TRB log file
 543661 asciilifeform_whogaveblox asciilifeform Record the origin of every incoming candidate block (whether accepted or rejected)
+606789 mod6_phexdigit_fix mod6 Fix decoding LUT which wrongly accepted certain invalid characters as hex
+606789 bitcoin_getrawtransaction jfw Add RPC to get transactions from memory pool or database (hex only)

The RPC itself is about as simple as it gets in this codebase. First we try the mempool, as this should be fast and may contain unconfirmed transactions.(i)

--- a/bitcoin/src/bitcoinrpc.cpp
+++ b/bitcoin/src/bitcoinrpc.cpp
@@ -1351,7 +1351,31 @@
     return entry;
 }

+Value getrawtransaction(const Array& params, bool fHelp)
+{
+    if (fHelp || params.size() != 1)
+        throw runtime_error(
+            "getrawtransaction <txid>\n"
+            "Get hex serialization of <txid> from memory pool or database.");

+    uint256 hash;
+    map<uint256, CTransaction>::iterator it;
+    CTransaction tx;
+    CDataStream ssTx;
+
+    hash.SetHex(params[0].get_str());
+    it = mapTransactions.find(hash);
+    if (it != mapTransactions.end())
+        tx = it->second;

Functions everywhere have to open their own database connections - though some have the luxury of getting one passed in - which then implies a whole caching mechanism so as not to be horribly inefficient. Odin knows why there couldn't just be one global (or at least per-thread) "This Is The Database; Use It" object.

+    else {
+        CTxDB txdb("r");
+        if (!txdb.ReadDiskTx(hash, tx))
+            throw JSONRPCError(-5, "Transaction not found in memory pool or database.");
+    }
+    ssTx << tx;
+    return HexStr(ssTx.begin(), ssTx.end());
+}
+
 Value backupwallet(const Array& params, bool fHelp)
 {
     if (fHelp || params.size() != 1)

Wiring the function into the RPC dispatch table (I don't recall how I chose where to insert it, as the list was already non-alphabetical; probably based on where it seemed sensible in the help listing):

@@ -1865,6 +1889,7 @@
     make_pair("getreceivedbyaccount",   &getreceivedbyaccount),
     make_pair("listreceivedbyaddress",  &listreceivedbyaddress),
     make_pair("listreceivedbyaccount",  &listreceivedbyaccount),
+    make_pair("getrawtransaction",      &getrawtransaction),
     make_pair("backupwallet",           &backupwallet),
     make_pair("keypoolrefill",          &keypoolrefill),
     make_pair("walletpassphrase",       &walletpassphrase),

The mempool object now needs to be visible between compilation units. I suggest doing a grep to verify this introduces no name conflicts.

--- a/bitcoin/src/main.cpp
+++ b/bitcoin/src/main.cpp
@@ -26,7 +26,7 @@

 CCriticalSection cs_main;

-static map<uint256, CTransaction> mapTransactions;
+map<uint256, CTransaction> mapTransactions;
 CCriticalSection cs_mapTransactions;
 unsigned int nTransactionsUpdated = 0;
 map<COutPoint, CInPoint> mapNextTx;
--- a/bitcoin/src/main.h
+++ b/bitcoin/src/main.h
@@ -46,6 +46,7 @@

 extern CCriticalSection cs_main;
+extern std::map<uint256, CTransaction> mapTransactions;
 extern std::map<uint256, CBlockIndex*> mapBlockIndex;
 extern uint256 hashGenesisBlock;
 extern CBlockIndex* pindexGenesisBlock;

I tested that it builds, successfully fetches transactions from both mempool and database, and returns the expected errors for missing argument or transaction not found. It does accept invalid hex strings, perhaps a flaw in that SetHex method. I've been running the patch in production since around August 10th of this year.

  1. The original cause for my writing the patch was a stuck transaction that wasn't getting relayed to miners or block explorers for unknown reasons. Upon fishing the raw tx from the mempool and submitting it to one such site, a useful error was finally obtained identifying the problem as the S-value normalization mess; the -lows option provided a workaround, after double-spending to self for safety, which was a whole other pain. [^]

2019-12-04

keksum, a Keccak implementation in C as standalone Unix utility: genesis

Filed under: Software — Jacob Welsh @ 17:36

I produced a Keccak implementation in May 2019, through about one week of intensive study and hacking. It builds on some techniques and routines I'd been developing for small, self-contained C programs on Unix, whereby the standard I/O library is thrown out in favor of a minimalistic interface fitting the needs of the program, requiring for portability only the system call wrappers, which generally have direct translations to assembly language. The approach ensures that system errors are detected without requiring effort by calling code and leaves no uncertainty regarding signal behavior, flushing of buffers or integer overflow. The resulting binary here (musl/amd64, static, unstripped) weighs in at 19K and contains not so much as a "malloc".

Regarding the permutation and sponge construction themselves, I found the Keccak reference quite comprehensible; the only topics I needed to brush up on were Linear Feedback Shift Registers and the polynomial algebra used to describe them.

Having previously written what ended up as possibly the world's slowest hash functions in an interpreted language, I was itching to make something fast for once: not pouring vast efforts into optimization but at least avoiding obvious slowdowns. I wish I could say I got it all right on the first try, or that I identified all mistakes through careful re-reading, but not quite:

jfw: well my keccak proggy hashes; now to look for some test vectors
jfw: sure enough I botched it:
jfw: mathematically, (x-1) mod y is equivalent to (x+y-1) mod y
jfw: (pretty much the definition of mod...)
jfw: but if x is an unsigned type -- which I made it, because it's used as an array index and those better be nonnegative -- the subtraction wraps *mod 2*
jfw: er, mod power-of-two
jfw: the arrays in question are indexed mod 5, which being coprime to any power of 2, gives a decidedly different result from if x were a signed or mathematical integer.
jfw: (in another part of the code I had in fact anticipated this.)

The arithmetic in question sure looked correct on the surface, so I tracked this down by adding fine-grained test probes to compare each step of the permutation against provided data.(i) Further contemplation lead me to see that the mod-5 operations were entirely invariant in the permutation's input and thus amenable to a lookup table without introducing a timing side-channel.

I did a full re-read a month after writing; the only mistake found was a comment with off-by-one range description.

Capacity and output length parameters are user-selectable; I chose a default of 512 bits for both, as explained in the code:

Sponge capacity is an upper bound on security level because the permutation is readily inverted if its full output is known. Beyond that, its relationship to actual security is not clear to me (or perhaps anyone); some margin of safety seems prudent. In the FIPS202 parameters, capacity under 512 is seen only in "SHAKE128" and none of the "SHA3" fixed-width hashes. The EuCrypt default is 256 (bitrate 1344); I have not found a discussion of this choice.

I should have piped up and asked about this at the time; though I was still a WoT non-entity living in the shadows, it happens that blog comments are one of the easier ways to get started in the grand conversation, and certainly if you have something interesting to say or ask.

The past being what it is, I will ask now: do any mathematical minds in the readership have input on what constitutes a "good" choice of capacity and why? If as I'd been thinking it's more than 256 to be at least as secure as SHA3, it would seem to suggest a need to regrind the existing Keccak vpatches or otherwise deal with a multiplicity of standards.

A compiler with 64-bit integer support is required; there is no dependence on machine byte order. The little-endian convention is used for interpreting bytes as the bit strings the permutation is defined on. I believe the code to be timing-invariant with respect to potentially secret bits, with the exception of output hex encoding. This would be good to fix. The other main deficiency I see is lack of a working "-c" option to verify provided hashes.

Finally, some basic performance numbers, from a modest Core 2 @2.26GHz, 3072K cache:

$ dd if=/dev/zero bs=1048576 count=100 | keksum
100+0 records in
100+0 records out
104857600 bytes (100.0MB) copied, 2.656227 seconds, 37.6MB/s
7a37879dcc634482775f4c1ebf294a0a02c9bcf924222cf6d2fe1c6beca3574debfe73f9034b868deefd7bbad4f5c251333bc3c735f1a82de045c7980814a2c2

$ dd if=/dev/urandom bs=1048576 count=100 > rand
$ dd if=rand bs=1048576 count=100 | keksum
100+0 records in
100+0 records out
104857600 bytes (100.0MB) copied, 2.648968 seconds, 37.8MB/s
ec4eab1cff9198e1ef23d663cc9da505eaeda713168bf31ed7807ec63fb1ddfa3454aea212fab193b071ad98e2cc47b5d004a4d1737d23ae8804978995ae1b7a

Download vpatches

  1. The spiffy modular types in Ada would have prevented this mistake entirely, though not without some cost either in runtime performance or compiler complexity (and thus potential bugs...), I would imagine. [^]

2019-12-03

Keccak background

Filed under: Bitcoin, Historia, Software — Jacob Welsh @ 18:52

"Keccak" is a cryptographic hash function, or rather, some primitives for constructing such functions in a desired size and shape, of relatively recent design as these things go. It was brought to the attention of the forum in early 2016 in the context of contemplating changes to the Bitcoin protocol,(i) (ii) (iii) and subsequently differentiated from SHA3.(iv)

Compared to the prevailing standards at the time - mostly variants on the MD4 concept, processing blocks of input through an iterated compression function - Keccak is based on a large pseudorandom permutation (1600 bits, though the spec also defines smaller variants). As this is readily invertible, the desired "one-way" property is provided by a "sponge construction", mixing in blocks of input and extracting output while iterating the permutation and keeping some number of its bits secret as internal state. This number is called the capacity (or by complement the rate, the two summing to the permutation bit width) and can be tuned for the desired balance of security and computational intensity. The construction can take unlimited input, or produce unlimited output as a kind of stream cipher.(v)

I started out in 2017 playing with a C implementation found in the wild, supposedly a "readable and compact" version written by the original team. With some cleanup I got it into a state that could be described as compact, but I couldn't get very far in reading it, at least without having first digested the spec. And it had the unfortunate limitation of requiring the full input and output to exist in memory, no streaming. My confidence as an applied cryptographer was growing and I soon implemented a number of classical hash functions, but set Keccak aside as not being an immediate necessity. Meanwhile, Diana Coman produced and incrementally published a very nice and documented reference implementation in Ada, which was adopted for use in V and soon became non-optional.

While I was well convinced by the Republican rationale for Ada, I was much less keen on introducing GNAT, the flagship implementation, into my environment. It was a million-plus-line-of-code beast that I wouldn't stand a chance to ever really understand; making matters worse, it was a "Thompsonism", a circular dependency requiring existing binaries in order to build from source and thus dubiously "open source" at all. While I already depended on one such thing - the C compiler - I was hoping to somehow keep this to ONE thing, or at least ensure a way to work with the crucial V on existing machines without pulling all this in.

Stay tuned for the result.


  1. mircea_popescu: actually i wouldn't go to war over keccak.
    mircea_popescu: letting bitfury & friends eat 100mn in unrecoupable engineering costs would provide exactly the correct lesson as to what it's a good idea to say and when it's a good idea to shut the fuck up and toe the line.

    [^]

  2. The necessary prerequisite for any change to the Bitcoin protocol [^]

  3. mircea_popescu: http://log.bitcoin-assets.com/?date=01-02-2016#1393026 << at least it wasn;t fucking developed by teh nsa.
    assbot: Logged on 01-02-2016 19:29:18; ascii_butugychag: ;;later tell mircea_popescu in what sense is adoptinc keccak a rejection of usg standards? it was actually adopted as sha3...
    mircea_popescu: as far as we know. whatevs. minor point.
    ascii_butugychag: btw between that thread and now i went and read the keccak spec
    ascii_butugychag: it is mighty spiffy.
    ascii_butugychag: accordionizes to size.
    mircea_popescu: :)
    mircea_popescu: i don't need to explain what i meant by not finite then ?
    ascii_butugychag: aha.
    ascii_butugychag: other hashes also accept infinite bits but they eat where they shit.
    mircea_popescu: quite.
    mircea_popescu: and mind that while in no means do i propose this is "Asic resistant", from a designer perspective you must appreciate i'm giving you a fun job to do.
    mircea_popescu: at least therer's that.
    mircea_popescu: always make sure everyone's having fun.
    ascii_butugychag: quite! nobody will be plagiarizing old verilog from fpga docs to bake this one.
    ascii_butugychag: very asian-resistant.
    ascii_butugychag: which is a mega-plus.

    [^]


  4. asciilifeform: holyshit the original keccak www is gone
    asciilifeform: replaced with some horrorshow
    asciilifeform: ada code -- gone
    asciilifeform: fortunately still on my hdd
    asciilifeform: check this out, keccak.noekeon.org now forwards to buncha tards
    asciilifeform: https://archive.is/GkmgU < original
    shinohai: Notice that happened after nist.gov declared their spec
    asciilifeform: shinohai: not immediately , iirc was still intact last yr
    asciilifeform: incidentally shinohai keccak != usg.sha3
    asciilifeform: they adopted ~particular params~ of keccak as the new national whatever
    asciilifeform: orig is ~family~ of functions.
    asciilifeform: see also https://archive.is/lViVh << since 'unhappened' article
    asciilifeform: ' The SHA-3 version of Keccak being proposed appears to provide essentially the same level of security guarantees as SHA-2, its predecessor. If we are going to develop a next generation hash, there certainly should be standardized versions that provide a higher security level than the older hash functions! NIST, in the original call for submissions, specifically asked for four versions in each submission, with at least two that would
    asciilifeform: be stronger than what was currently available, so it's hard to understand this post-competition weakening.'
    asciilifeform: didjaknow.
    asciilifeform: notice how 'everyone' nao thinks 'oh, keccak? that's called sha3 nao' [^]
  5. Since state is still finite, output will of course repeat eventually; one would hope this cycle length approaches that order of 21600. [^]

2019-12-02

Mapping context and scope for a Keccak writeup

Filed under: Software, Writing — Jacob Welsh @ 19:42

I intended to briefly introduce and present my implementation of the Keccak family of cryptographic hash functions today. A relatively small and self-contained piece of code, doing a well-defined thing, solving a well-understood problem... should be easy, right? Or so I thought, for about two minutes.

As I considered what context would be necessary even for readers already educated in "wtf are hash functions?", it soon blew up into a whole web of connected topics: some already written up elsewhere, some clear enough in my head to write up now, some needing a refresher, and some requiring further research.

There'd be history of Keccak itself; history of its interest to the forum, including why not SHA3; the seeming lack of a spec in place of SHA3 for "last hundred yards" questions like byte order and parameter selection; the degree to which the original paper does or doesn't help with those questions; history of interest in the Ada programming language in the forum; how I wanted to embrace it but was averse to accepting GNAT; approaches I made to that problem; an attempt I made to tidy up an older Keccak implementation; deciding to do my own in C; the level I aim to work at in that language regarding support libraries and why; my experience of reading the Keccak paper and filling in knowledge gaps; my observations in comparing it with SHA3; how I approached the parameter question and what remains open there; the process of implementing, mistakes made and tracking them down. Then could come presentation of the code itself, noting requirements, tested platforms and observed performance.

Perhaps I've now exaggerated the problem a wee tad; surely something sensible could be produced with a scope in between "everything that could possibly be said" and "here's some code: good luck". Certainly the debts of my past writing avoidance are making themselves felt. I hope I've at least better illuminated for myself a path out of the pit and provided a hint of things to come. Onwards!

2019-12-01

Gales Linux initial release

Filed under: Software — Jacob Welsh @ 21:05

Today I am pleased to present, at long last, an initial public release of my Gales Linux project.

The goods

To get started you may wish to examine the files:

  • BUILD, a lengthy recipe for bootstrapping the system from source.
  • INSTALL, listing the steps to install the result on a machine. Presently quite terse; some experience with manual Linux installation such as in Gentoo will likely be needed as well as some clarifications.
  • PORTS, explaining how to get started working with gports and turn the rather minimal base system into something more comfortable.
  • gports/gales-util/gales-mirror-sync, with which you can download the much larger collection of sources referenced from this repository, both base and ports.(i) (ii)

Mirror sync is designed to work with minimal dependencies: you should be able to copy the script from the repository and run it on an existing Unix-like system, setting the DST variable to a fresh download directory and possibly replacing the "wget" call with "curl" or your preferred download tool. HTML scraping is neither required nor supported. If you'd rather not use my script you could parse the manifest yourself. (Just don't go putting something that blindly re-downloads the whole thing in a cron job, m'kay?) The hashes in the manifest act as a first pass of error detection in storage and transit, while canonical ones are in the signed repository.

The full mirror presently weighs around 440MB. Distribution is quite skew, with the top 10 file sizes:

$ awk '{print $2,$1}' Manifest.sha512 | sort -nr | head
93192404 linux-4.9.tar.xz
82935453 gcc-4.7.4.tar.bz2
76156220 gcc-6.4.0.tar.xz
22887305 db-4.8.30.tar.gz
22716802 binutils-2.24.tar.bz2
12495628 Python-2.7.13.tar.xz
12465748 php-5.6.34.tar.xz
11961692 perl-5.26.0.tar.xz
11570420 perl-5.24.2.tar.xz
9377313 bash-4.4.tar.gz

You'll need to supply the Linux kernel patch of your choice, or the whole thing if you need something other than 4.9.

The ports collection consists of:

acpica autoconf automake bash bc bison bzip2 cl-hyperspec clisp dash db flex gales-util gcc64 git gnupg less libevent libressl libusb links m4 man-pages man-pages-posix mandoc ncurses nginx ocaml openssh patch pciutils perl php56 py-setuptools python python-docs qmail readline redis sbcl sqlite sqlite-doc tmux ucspi-tcp vim xz zlib

Note that some of these may be trimmed down or configured in more minimal ways than you're accustomed to, either due to compatibility problems with musl or static linking or as a general precaution. I'm not opposed to re-enabling features upon reasonable demonstration of safety, or you can certainly edit the builds to suit your own needs. Things I consider high priorities for porting effort include:

apache emacs gdb ghc gnat hdparm iptables mysql sdparm shred smartctl strace texinfo X11

Obligatory disclaimer: Like any operating system targeting the "modern" hardware and software cocktail, Gales Linux contains large amounts of toxic and hazardous materials both known and unknown. While I have striven to make prudent and security-conscious choices, I am not attempting to keep up with the "penetrate and patch" rat-race in its many third-party components. Be careful out there.

Next steps

As my goal for now has been to release what I have without too much further fiddling, a few obstacles remain for a proper V genesis of the repository.(iii) What I've noted so far are some executable scripts in the repository that would either need to be invoked differently or somehow "chmod"ed before use, and one patch (to bin86) that includes a control character in its context lines. Also some clutter for potential removal is a number of old-style "context diffs" that I reground to unified format for BusyBox compatibility but preserved the originals for the sake of preservation (e.g. in bash and vim ports).

  1. I've fixed the previously noted subdirectory flaw. [^]
  2. Present mirror IP: 198.199.79.106 welshcomputing.com [^]
  3. If such a direct conversion is even a sensible way to go; for instance, including externally maintained items as tarballs by hash reference is not in keeping with V principles, but taking ownership of the whole mess will be a larger project. [^]

Powered by MP-WP. Copyright Jacob Welsh.