[Programming] Kafkaesque rabbits in the queue at the pub

Orca

Kakodaimonos wrote: »

That was the function.

The annoying ones are the feeds that decide to do a ulong with only 6 bytes.

But that's my point. Maybe you've got a string of bytes. You cast it into a header, then use the header to start decoding everything else. And for that, you do read<offset, length, type>(buffer). At no point do you manually do any shifts and masks unless you're writing the original read function.

Or maybe you've got a deserializer that has the buffer internally, and then it's a matter of doing a read<length, type>() and it advances an internal pointer into the buffer.

Once you've verified your read function works correctly, then it's a simple matter of looking at the code using it, and making sure the types are correct, offsets, lengths, etc. are correct. And for those, it may be possible to use code generation so even that is sure to be right after you've done it the first time.

I look at manual bit manipulation very suspiciously, unless you're doing something that's a one-off for a specific (internal) purpose. It's like pointer arithmetic. Yes you can, but is there a more maintainable way to do it that has similar performance characteristics?

crimsoncoyote

It's been a while, programming thread.

Here's my latest complaint on what I'm working with:
Java (I know I know) code written by c devs... It's painful.

Lots of stuff like:

public class Foo
{
    private String m_s;
    //...
    public Foo (String s)
    {
         m_s = s;
    }
    //...
}

=/

Also I'm tasked with making the JNI changes for this thing we're doing and I'm not really looking forward to that.

bowen

Wait I thought that was how you're supposed to do it in Java.

Did Java finally get auto properties like c#?

hippofant

bowen wrote: »

Wait I thought that was how you're supposed to do it in Java.

Did Java finally get auto properties like c#?

I suspect the complain might be about the names m_s and s. Not sure if those have been obfuscated though.

bowen

there's no public accessors I guess

but I'd argue you might not need those for internal variables that don't/shouldn't be changed

like if it's immutable like a fraction's numerator or denominator

Jasconius

this is

hilarious

https://arstechnica.com/security/2017/03/firefox-gets-complaint-for-labeling-unencrypted-login-page-insecure/

get rekt

mightyjongyo

"It's never been hacked in 15 years!!"

* gets hacked *

ed: seriously, what did you expect people to do after making a statement like that...

bowen

it's less, hasn't been hacked, to, there's been no interesting in hacking your shitty website

hippofant

bowen wrote: »

there's no public accessors I guess

but I'd argue you might not need those for internal variables that don't/shouldn't be changed

like if it's immutable like a fraction's numerator or denominator

I assume there's more code under //...

crimsoncoyote

It's the naming convention, lack of "this" in the constructor (I know it's not required, but I like it there), and that it's assigning the reference to the object and not creating a new one (they do it for more than just Strings and are inconsistent with when they use)

Edit: Yeah string was a bad example, sorry about that

bowen

I can get behind that.

Different paradigms though. OOP is hard to master coming from C.

crimsoncoyote

On the bright side, I don't have to use any of the already-written stuff.

Unfortunately my Java is pretty rusty.

dporowski

Okay, this seems a good place.

Any advice for getting better with whiteboarding? Put me in front of an IDE or something, and I can noodle around and make things happen with minimal issue. Put me in front of a board, electronic or physical, and I may as well be using a crayon and/or my own poop. And I mean one of the fat crayons they give kids who might try to put it in their nose, not like a good Crayola or something.

It's its own skillset, I know that, but seems a little impractical to practice... Could be wrong there though.

hippofant

NM. :redface:

admanb

dporowski wrote: »

Okay, this seems a good place.

Any advice for getting better with whiteboarding? Put me in front of an IDE or something, and I can noodle around and make things happen with minimal issue. Put me in front of a board, electronic or physical, and I may as well be using a crayon and/or my own poop. And I mean one of the fat crayons they give kids who might try to put it in their nose, not like a good Crayola or something.

It's its own skillset, I know that, but seems a little impractical to practice... Could be wrong there though.

Maybe grab a bunch of common interview problems and then practice typing pseudocode algorithms into a plain text editor? That's pretty close to it.

The main trick to whiteboarding is to think out loud and constantly communicate with your interviewer. I very frequently ask, "can I just assume we have a helper method that does X?" or "I know JS has a method that does Y but I don't remember what it's called" and 90% of the time they just nod and say we can assume that exists.

dporowski

admanb wrote: »

dporowski wrote: »

Okay, this seems a good place.

Any advice for getting better with whiteboarding? Put me in front of an IDE or something, and I can noodle around and make things happen with minimal issue. Put me in front of a board, electronic or physical, and I may as well be using a crayon and/or my own poop. And I mean one of the fat crayons they give kids who might try to put it in their nose, not like a good Crayola or something.

It's its own skillset, I know that, but seems a little impractical to practice... Could be wrong there though.

Maybe grab a bunch of common interview problems and then practice typing pseudocode algorithms into a plain text editor? That's pretty close to it.

The main trick to whiteboarding is to think out loud and constantly communicate with your interviewer. I very frequently ask, "can I just assume we have a helper method that does X?" or "I know JS has a method that does Y but I don't remember what it's called" and 90% of the time they just nod and say we can assume that exists.

Yeah, I've been thinking about doing that anyway. I just am absolutely trash at thinking out loud; the bits of my brain that do "thought" seem to be the same ones that do "communicate with other humans". Practice, practice, etc I suppose.

I just get annoyed by it; "how many weird things have you memorized" doesn't test how well you can do anything but answer interview questions. "Yeah hang on I'll just look that up 'cause everyone asks how to do it and I don't want to reinvent the wheel" just doesn't fly though.

admanb

Oh yeah they're utter garbage. Of the many, many interview segments I've done over the last six months I'd say only 10-20% of them any more valuable than a shitty, stressful way to make sure I knew the basicest basics of coding.

Grape Ape

Best technical interview I've had to date was essentially "you have 3 hours and open access to the internet, gives what you have for a solution to this problem"

Phyphor

Stupid Windows Tricks

Lets say you want to fake a device for simulation in windows and you want to trap accesses but still allow code to pretend it's MMIO. You can abuse exception handling do that! (on x86 anyway)

struct Device
{
	HANDLE h;
	uint64_t Public;
	uint8_t *Private;
	uint64_t Size;
} dev;

struct TrapInfo
{
	uint64_t addr;
	uint64_t old_data;
	DWORD old_protect;
	DWORD flags;
};

void OnRead(uint64_t offset) { printf("RD @%p\n", offset); }
void OnWrite(uint64_t offset, uint64_t before, uint64_t after) { printf("WR @%p: %p to %p\n", offset, before, after); }

long FancyWindowsExceptionHandler(DWORD code, LPEXCEPTION_POINTERS ptrs, TrapInfo *trap)
{
	uint64_t addr = ptrs->ExceptionRecord->ExceptionInformation[1];

	switch(code) {
	case EXCEPTION_ACCESS_VIOLATION:
		if(addr >= dev.Public && addr < dev.Public + dev.Size) {
			ptrs->ContextRecord->EFlags |= 0x100; // Set CPU-level single-step
			trap->addr = addr;
			// Set permissions to allow access for the next instruction to run
			if(ptrs->ExceptionRecord->ExceptionInformation[0]) {
				// Write
				VirtualProtect((void*)addr, 8, PAGE_READWRITE, &trap->old_protect);
				memcpy(&trap->old_data, (void*)addr, 8);
				trap->flags = 1;
			} else {
				// Read
				VirtualProtect((void*)addr, 8, PAGE_READONLY, &trap->old_protect);
				trap->flags = 0;
			}
			return EXCEPTION_CONTINUE_EXECUTION;
		}
		break;
	case EXCEPTION_SINGLE_STEP:
		ptrs->ContextRecord->EFlags &= ~0x100U; // Clear the single step flag
		// Notify "device" of what happened
		if(trap->flags) {
			uint64_t current;
			memcpy(&current, (void*)trap->addr, 8);
			OnWrite(trap->addr, trap->old_data, current);
		} else {
			OnRead(trap->addr);
		}
		VirtualProtect((void*)trap->addr, 8, trap->old_protect, &trap->old_protect);
		return EXCEPTION_CONTINUE_EXECUTION;
	}
	return EXCEPTION_CONTINUE_SEARCH;
}

void main()
{

	uint32_t size = 4096;
	// Create the device memory - a private set the implementation can use + a public set that will be trapped
	dev.Size = size;
	dev.h = CreateFileMapping(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 0, size, NULL);
	dev.Private = (uint8_t*)MapViewOfFile(dev.h, FILE_MAP_READ | FILE_MAP_WRITE, 0, 0, 0);
	dev.Public = (uint64_t)MapViewOfFile(dev.h, FILE_MAP_READ | FILE_MAP_WRITE, 0, 0, 0);
	DWORD old;
	VirtualProtect((void*)dev.Public, size, PAGE_NOACCESS, &old); // or READONLY if not read-significant

	uint64_t *device_mem = (uint64_t*)dev.Public;

	TrapInfo trap;
	__try {
		device_mem[0] = device_mem[1] + 1;
	} __except(FancyWindowsExceptionHandler(GetExceptionCode(), GetExceptionInformation(), &trap)) {}
}

Prints
RD @00000000000E0008
WR @00000000000E0000: 0000000000000000 to 0000000000000001

Alistair Hutton

Anyone got any suggestions for software/libraries for doing high dimensional data clustering. Looking at vectors with around ~10,000 dimensions almost all of which will be binary 1,0 but a few (like less than a dozen) with continuous data

schuss

dporowski wrote: »

admanb wrote: »

dporowski wrote: »

Okay, this seems a good place.

Any advice for getting better with whiteboarding? Put me in front of an IDE or something, and I can noodle around and make things happen with minimal issue. Put me in front of a board, electronic or physical, and I may as well be using a crayon and/or my own poop. And I mean one of the fat crayons they give kids who might try to put it in their nose, not like a good Crayola or something.

It's its own skillset, I know that, but seems a little impractical to practice... Could be wrong there though.

Maybe grab a bunch of common interview problems and then practice typing pseudocode algorithms into a plain text editor? That's pretty close to it.

The main trick to whiteboarding is to think out loud and constantly communicate with your interviewer. I very frequently ask, "can I just assume we have a helper method that does X?" or "I know JS has a method that does Y but I don't remember what it's called" and 90% of the time they just nod and say we can assume that exists.

Yeah, I've been thinking about doing that anyway. I just am absolutely trash at thinking out loud; the bits of my brain that do "thought" seem to be the same ones that do "communicate with other humans". Practice, practice, etc I suppose.

I just get annoyed by it; "how many weird things have you memorized" doesn't test how well you can do anything but answer interview questions. "Yeah hang on I'll just look that up 'cause everyone asks how to do it and I don't want to reinvent the wheel" just doesn't fly though.

Every time you come up with a solution, have a piece of paper next to you and sequence it/build a picture. Whiteboard is just a larger scale. Another key to whiteboarding is knowing your audience and using pictures/code or text as needed for the audience to understand. Try to develop a keen sense for what people are responding to, as that will help close the communication gap.
Most of the time I just use pictures/boxes with process flows (not a developer, so grain of salt) and lots of arrows. With some folks though, it's just a few arrows, a word here and there and random half-pictograms as we're on the same page.

Grape Ape

Alistair Hutton wrote: »

Anyone got any suggestions for software/libraries for doing high dimensional data clustering. Looking at vectors with around ~10,000 dimensions almost all of which will be binary 1,0 but a few (like less than a dozen) with continuous data

PCA to reduce that down if it's any kind of sparse

I think lasso is strong choice for very wide data. At least, it gets thrown around in a compbio setting a lot.

djmitchella

Looking through history of some bits of code, and these lines (thankfully) got removed:

var ow = service.getOwnerFilterIndex();
ctrl.setOwnerFilter(ow && ow > 0 ? ow : 0)

That second line feels more like a (bad) programming interview question than I've seen in a while (and the answer to "what does this code do" is "get cleaned up a lot so that mystery precedence bugs don't come up any more, sheesh, and does it even make sense in the first place?"). It also doesn't help that the variable is literally called "ow", which is what that code made me say.

urahonky

djmitchella wrote: »

Looking through history of some bits of code, and these lines (thankfully) got removed:

var ow = service.getOwnerFilterIndex();
ctrl.setOwnerFilter(ow && ow > 0 ? ow : 0)

That second line feels more like a (bad) programming interview question than I've seen in a while (and the answer to "what does this code do" is "get cleaned up a lot so that mystery precedence bugs don't come up any more, sheesh, and does it even make sense in the first place?"). It also doesn't help that the variable is literally called "ow", which is what that code made me say.

What if ow had a chance to be less than 0 and any negatives broke it?

bowen

urahonky wrote: »

djmitchella wrote: »

Looking through history of some bits of code, and these lines (thankfully) got removed:

var ow = service.getOwnerFilterIndex();
ctrl.setOwnerFilter(ow && ow > 0 ? ow : 0)

That second line feels more like a (bad) programming interview question than I've seen in a while (and the answer to "what does this code do" is "get cleaned up a lot so that mystery precedence bugs don't come up any more, sheesh, and does it even make sense in the first place?"). It also doesn't help that the variable is literally called "ow", which is what that code made me say.

What if ow had a chance to be less than 0 and any negatives broke it?

it'd probably hurt

Orca

I've become inordinately fond of intermediate variables, e.g.

auto rc = do_lengthy_operation();
bool timed_out = rc == ReturnCode::has_timed_out;
auto state = get_state();
bool can_retry = (state == InternalState::StateAllowingRetry1) or (state == InternalState::StateAllowingRetry2);

if (timed_out and can_retry)
{
    rc = do_lengthy_operation();
    ...
}

Even something like the above would be helped with a function, e.g.:

var owner = service.getOwnerFilterIndex()
ctrl.setOwnerFilter(clip(owner, 0, MAX_FILTER_INDEX))

The bigger issue is why can you have negative indices that aren't okay? That seems like an invalid state to be in that probably should be fixed instead of papered over.

Dehumanized

love when the code literally reads ow ow ow

LD50

var owner = Math.abs(service.getOwnerFilterIndex())

Fixt.

Dehumanized

id just factor that right on down to

ctrl.setOwnerFilter(service)

[Michael]

http://stackoverflow.com/insights/survey/2017/
Long but sorta interesting. It's got some super insightful results, like:

A vast majority—65.6%—of developers pronounce 'GIF' with a hard 'g,' like gift. We're not sure how the 2% of developers who chose "some other way" say it, but we're very, very curious.

bowen

[Michael] wrote: »

http://stackoverflow.com/insights/survey/2017/
Long but sorta interesting. It's got some super insightful results, like:

A vast majority—65.6%—of developers pronounce 'GIF' with a hard 'g,' like gift. We're not sure how the 2% of developers who chose "some other way" say it, but we're very, very curious.

gwif

djmitchella

LD50 wrote: »

var owner = Math.abs(service.getOwnerFilterIndex())

Fixt.

I don't think that's it:

> var ow = 0;
> (ow && ow > 0 ? ow : 0)
< 0
> ow = 10;
> (ow && ow > 0 ? ow : 0)
< 10
> ow = -10;
> (ow && ow > 0 ? ow : 0)
< 0

I think it's actually Math.max(0, ow), which I guess makes sense if the getOwnerFilterIndex call was returning -1 for "not in the list". But that doesn't explain the initial "ow &&", or how it could end up with that in there. This is all many changesets ago, though, so possibly it just got written wrongly and then fixed up in a later commit. Given how operator precedence works, it's doing:

( ow && (ow > 0) ) ? ow : 0

but this is Javascript, so there's also casting thrown into the mix there. Let's see.

> 10 && true
< true
> 10 && false
< false

so the && doesn't seem to be doing much in practise, either. (fun with Javascript: if I flip the && around, I get this)

> true && 10
< 10
> false && -10
< false

LD50

Yeah, I knew the answer wasn't 'right'. I mean, even if it was, would you want to return some random incorrect index when it was given an invalid negative one?

urahonky

Yep it is definitely "gif" as in "gift". I don't give a damn what the guy who invented it says.

bowen

urahonky wrote: »

Yep it is definitely "gif" as in "gift". I don't give a damn what the guy who invented it says.

what about them lacers, watch your eyes around them.

Don't forget to enter your pine at the ATM.

djmitchella

[Michael] wrote: »

http://stackoverflow.com/insights/survey/2017/
Long but sorta interesting. It's got some super insightful results, like:

A vast majority—65.6%—of developers pronounce 'GIF' with a hard 'g,' like gift. We're not sure how the 2% of developers who chose "some other way" say it, but we're very, very curious.

I always wonder how people pronounce .exe:

as per the late lamented .EXE magazine, it's pronounced to rhyme with "not sexy (magazine)"

(original home of Verity Stob; I can't find any of her articles from that era online, but here's a classic from Dr.Dobbs

Verity Stob has developed a new tool that will help you make rapid diagnoses of sick PCs. A rolling computer gathers "cruft." When you spot a class interface that is no longer used by any client, but that nobody dare delete, that's cruft. It is also the word "seperate," added to a spellchecker's private dictionary in a moment of careless haste, and now waiting for a suitably important document. Cruft is the cruel corruption and confusion inevitably wrought by time upon all petty efforts of humankind. There.

At Laboratoires Stob, we have been working on the cruft crisis for a while. Recalling the maxim "to control a problem you must first measure it," we have devised a suitable metric, an index of cruftidity. Our first version, presented below, is based on a typical PC installation running Windows 2000. But there will shortly be ports to Linux, Mac OS X, and other Unices; we are confident these OSes are just as prone.

We would like to acknowledge our debt, in the construction of this instrument, to Rear-Admiral Sir Francis Beaufort. His 1805 scale of windspeeds ("Insurance Claim Force 8. Description on land: Tile blown off roof falls onto litigious neighbour's Toyota Shiny") is as valid and useful today as it ever was. Enough preamble.

Cruft Force 0. Virgin. Description: The "Connect to the Internet" shortcut is still on the desktop, and the "How to use Windows" dialog appears at logon. Menu animations and the various event-based sound effects — even the dreaded Microsoft Sound — seem cheerful and amusing. Likewise, a clandestine installation of the Blue Screen Of Death screensaver (complete with simulated reboot, natch) from the Sysinternals web site is hilarious. Compilers run crisply, and report only sensible, easily resolved errors. There are just nine directories off C:\.

Filled with the enthusiasm that goes with having a brand new machine, the user resolves to stick to the new-fangled security-conscious temp directory buried deep somewhere below Documents and Settings.

Cruft Force 1. New. Description: User has taken time to rename cutesy desktop icons incorporating the first person singular possessive pronoun.

Twice, the mouse cursor has done that poltergeist trick where, with the actual mouse stationary, it drifts three inches due east and then stops. For no reason at all. Works fine afterwards though. Brrrrrrr.

Cruft Force 2. Comfortable. Description: User has now got around to resetting Explorer so that "web content in folders" is suppressed. Something has made a C:\TEMP directory in the proper place unasked, for which mercy the user guiltily feels grateful.

A strange entry is found in the System event log: MRxSmb: The redirector was unable to initialise security context or query context attributes. Assiduous googling of the key phrases, up web site and down newsgroup, establishes that, although many have wondered, nobody knows what this means.

Cruft Force 3. Lived-in. Description: One time in seven when the user starts Word or other Office 2000 app, instead of running, it pretends it is installing itself for the first time and starts a setup program.

Directory count in C:\ up to 17, and something has pooed a Paradox lock control file there, too.

Cruft Force 4. Middle-aged. Description: Amount of time from screen showing "real" Windows background to the logon box appearing is >30 seconds. Sometimes cannot "browse" other machines on LAN.

Get first real BSOD. Uninstall jokey screen saver, replace with SETI.

An extra disk of huge capacity has been installed. CD-ROM moves from drive F: to drive [:

Cruft Force 5. Worn out. Description: Some time after bootup, always get a dialog "A service has failed to start - BLT300." What is BLT300? Nobody knows. Although one can manually remove/disable this service, it always reappears two or three reboots later.

If one double-clicks a document icon, Word takes 4 minutes 30 seconds to start up. But it still works fine if started as a program. Somebody opines that this is due to misconfigured DDE. Or the Mars-Jupiter cusp.

Cruft Force 6. Limping. Description: [Delphi|Visual Basic|Java] suddenly remembers a trial shareware component — deleted six months ago because it was rubbish — and refuses to compile anything until it is reinstated.

"Web content in folders" Explorer setting switches itself back on unbidden. "Setup" programs start crashing while unpacking their own decompression DLLs.

Cruft Force 7. Wounded. Description: No longer able to logon using original account as the system freezes, so must logon as "Verity2" or similar.

There are now nine items in BOOT.INI: the original W2K starter, a brace of two-entries-each NT4s (one Turkish), a Windows 98, and three assorted Linuxen. Left to start up by itself, the machine chooses a broken installation of SUSE and halts with a kernel panic.

Cruft Force 8. Decrepit. Description: A virus checker is installed at the insistence of IT. This actually improves performance, apparently violating Newton's laws.

Blue Screens Of Death are served daily. The SETI screen saver, like ET himself, encounters difficulty calling home and despairing during an overnight run creates 312 copies of its icon in an (impressively expanded) system tray that fills half the screen.

Successful connections to the LAN are very rare.

Cruft Force 9. Putrefaction. Description: Can only see the 32-GB \ partition — the one which has all the source code on it — at every third boot. Directory count in C:\ up to 93, partly because some [one/thing] has put a complete (but non-working) installation of the Eudora e-mail client in the root.

Starting Control Panel shows rolling torch animation. The applet icons never appear.


Cruft Force 10. Expiry. Description: Machine only runs in Safe mode at 16-color 800×600, and even then for about a minute and a half before BSODing. Attempts to start an app are rewarded with a dialog "No font list found."

Ordinary dodges, such as reformatting the hard disk(s) and starting again, are ineffective. Cruft has soaked into the very fabric of the machine, and it should be disposed of safely at a government-approved facility. There it will be encased in cruft-resistant glass and buried in a residential district.

Phyphor

ee-ex-ee

urahonky

I love getting reports that users are complaining about broken functionality after a release. "When I click this button it doesn't print anymore. Do you guys even test your releases?" and only to find out that there was a request to remove said functionality that went into affect months ago (thank you git history!).

Alistair Hutton

Grape Ape wrote: »

Alistair Hutton wrote: »

Anyone got any suggestions for software/libraries for doing high dimensional data clustering. Looking at vectors with around ~10,000 dimensions almost all of which will be binary 1,0 but a few (like less than a dozen) with continuous data

PCA to reduce that down if it's any kind of sparse

I think lasso is strong choice for very wide data. At least, it gets thrown around in a compbio setting a lot.

Oh yeah, this shit is getting PCA'd as it is helluva sparse.

I've been trying scikitlearn and it seems to have everything i need.

Spawnbroker

urahonky wrote: »

I love getting reports that users are complaining about broken functionality after a release. "When I click this button it doesn't print anymore. Do you guys even test your releases?" and only to find out that there was a request to remove said functionality that went into affect months ago (thank you git history!).

My favorite was a client that wanted users to be able to view a document through the web browser, but not be able to print it out.

I had to try and stay calm as I asked them what if the user just takes a screenshot, but they didn't know what I was talking about. They just really did not want the users to print their precious documents, and I'm like BUT IT'S ON THEIR SCREEN, THEY CAN JUST USE SNIPPING TOOL IF THEY WANT