GnomeTankWhat the what?Portland, OregonRegistered Userregular
That's random RGB, which is going to generate truly random clown colors. I want the HSL solution, because I want to create a range of colors that "match", and aren't all clown car esque. Thanks for the link Bowen.
First time I've needed to use beautiful soup, but it almost certainly saved me a lot of hassle. I'm supposed to write a deal which will log into an ftp, download an xml file, parse it, pull some data out, and then do stuff with it. Nothing too crazy, right? I download the xml to see what the format is as I have no documentation on that. The first thing my I catch are <font> tags. This is actually a broken html document which has <html>, <meta>, <title> (not inside a <head>), <body>, some old school html 4 <center> and <font> tags, a couple <br> and then suddenly an xml doc starts with no dtd. I get to the bottom of it... there's a closing xml tag with no matching opening tag and then no closing tags for the html.
beatuiful soup managed to make sense of it and let me pull out the not-broken part of the xml and get to work anyway.
I'm sorry I initiated the Polish Invasion ruleset by accident somehow.
not a doctor, not a lawyer, examples I use may not be fully researched so don't take out of context plz, don't @ me
0
jackalFuck Yes. That is an orderly anal warehouse.Registered Userregular
edited May 2012
I have that problem within my own company.
Them: "I think your program is making other programs hang up."
Me: "It shouldn't even effect it. It's hanging up for me too."
Me: "I attached a debugger and it is hanging up inside *enterprise security thingie that intercepts windows api calls." It seems to be caused by a race condition."
Them: "I don't know what a 'debugger' is or a 'race condition' and we can't uninstall *enterprise security thingie that intercepts windows api calls" because it is company policy to have it installed. Since I can't understand or test your explanation I'm going to just rebuild the users' computers and have meetings for weeks. Thanks though."
I think I need to find a new job. We currently have at least three levels of "enterprise security thingie" from various companies and no one thinks that is why absolutely everyone's computer runs slow and hangs randomly.
jackal on
0
GnomeTankWhat the what?Portland, OregonRegistered Userregular
Nothing tickles the wick like bad Windows API hooks.
I spent about a week looking at and had a bunch of guys in Florida spend a few days getting a remote kernel debug session set up for me so that I could diagnose a system freeze occurring while using one of my drivers. Actual cause? A popular AV had a driver that was hanging while intercepting network i/o coming from our driver
not a doctor, not a lawyer, examples I use may not be fully researched so don't take out of context plz, don't @ me
0
jackalFuck Yes. That is an orderly anal warehouse.Registered Userregular
No one suspects the enterprise software that they paid literally millions for (I don't fucking know how. when I ask for Balsamiq for my team they act like 500 dollars would break the bank) that is wrapped around the OS like a giant squid. It must be the absolutely vanilla .Net app. I get the impression that people got mad that I even suggested it. Sunken cost fallacy is apparently really strong.
Document, take screenshots, whatever the hell you need to do.
I told the guy, "You are honestly lucky I can't bill you for my services because this thing that took me 5 minutes to do and you 8 hours, and still unresolved, yesterday should be charged at a premium."
not a doctor, not a lawyer, examples I use may not be fully researched so don't take out of context plz, don't @ me
I honestly can't think of a library that makes me happier than Beautiful Soup. It makes a task that would otherwise be horrible and tedious simple and, dare I say it, fun. It is wonderful.
No one suspects the enterprise software that they paid literally millions for (I don't fucking know how. when I ask for Balsamiq for my team they act like 500 dollars would break the bank) ...
Well, the guys who sell the $500 software aren't going to wine and dine the CTO, fly him to a conference, and take him golfing...
Ok so the select syntax for Linq makes it impossible to debug issues when messing with XML... I had to rewrite the code out in a way where I could see what IEnumerable was being returned until I finally figured out I needed to add the namespace when filtering Elements.
Ok so the select syntax for Linq makes it impossible to debug issues when messing with XML... I had to rewrite the code out in a way where I could see what IEnumerable was being returned until I finally figured out I needed to add the namespace when filtering Elements.
Yeah, you've stumbled upon the only weakness I've found when dealing with LINQ so far.
Of course, I tend to think that chains of LINQ calls that go more than, say, 4 deep, are a code smell anyway, so I will park them in intermediate variables during initial development. If I have a chain of calls larger than that I actually like to refactor that out into a named method that expresses what the intent of chaining all that together is, just for the sake of readability.
EDIT: Wait, now that I think about it, isn't there a way to load your XDocument with the namespace so you don't have to worry about that? Yeah, this call should help.
Can anyone recommend a data collection type to me? I want something like a Map, but I want each key to be able to have multiple values. Like if I'm making a Dictionary class, I want each word to be able to have one or more definitions. Currently I have a Map where the keys are all tied to a Vector, but it seems very inelegant and I was hoping for something a bit smoother. Ideally I'd like to give them an upper limit, so after putting, say, 5 values in, adding another will delete one of the current entries, just like when you add a new value to an existing key in a map.
If it's not possible I guess I'll just have to work a little more logic into the Vector, and probably tie it all up in a new class that extends a Map, but that's sounding like a complicated solution to my one key -> multiple values problem.
You're looking at a multimap, ie key-value without a uniqueness constraint. As far as I'm aware none will get you "only store last X," that's almost a cache right there
I hadn't heard of them before, but multimaps seem to be maps where the value is a collection, is that right? If there's no way to limit it to storing, say, 5 values, I might be better of making my own collection to work the way I want then.
(I'm trying to store most commonly used words, with the first letter as the key, but I want the file to not get stupidly big, and only storing the last 5 or 10 keeps the file small, and also purges uncommon words)
It depends, the implementations I've seen just treat them as co-equal nodes in the data structure. insert() just always inserts, never overwrites and erase() typically needs a specific node or deletes the first one with that key. I've never seen a limit or a second collection style one.
For what you're trying to do, how would that work anyway? How do you decide to evict anything without keep track of a lot more? Plus, storage for say, q x z y v w could be better used expanding other more common letters and so on
I hadn't heard of them before, but multimaps seem to be maps where the value is a collection, is that right? If there's no way to limit it to storing, say, 5 values, I might be better of making my own collection to work the way I want then.
(I'm trying to store most commonly used words, with the first letter as the key, but I want the file to not get stupidly big, and only storing the last 5 or 10 keeps the file small, and also purges uncommon words)
That is a map that stores fixed-length deque objects as its values.
Make a deque class that allocates an object array of size X, you keep an index value to show you where the next insert is and just fill it up as you go. Once you get over X you naturally wrap around, and you have a full circular buffer. The container structure is static so it is about as efficient as you'll get, you just naturally overwrite the oldest stored value as you go.
That sounds good, and Java has an ArrayDeque class that seems to do what I'd need. The biggest limiting factor I have is that I need to perform multiple calculations on each value within about 0.2 seconds, because I need it to appear instantaneous, or as close as possible, and this is in Java so speed is definitely an issue. Having 10 entries under X or Z is probably unnecessary, but I can't make up for it with 20 entries under E because then every time I search for E I'll get some slowdown, and with E being so common it's the last thing I want slowdown for.
Would ArrayDeque.push(element) do that circular buffering? Or will I need to implement my element adding specially?
The picture of a Trie on wikipedia makes it look very useful, thanks. Either that or Deque should do it.
I know 0.2 seconds is a lot, but if I'm tracking every key and every word a user types, and trying to predict the word they're typing and the sentence or sentence fragment, I've got a lot of overhead attached to each keypress in the program, and I need to keep it quick. If I just stored every word the user entered it wouldn't be long before it's trying to work out which of the 8000 words starting with T has been used the most, and if there's even a tiny delay on keypress then there are going to be an awful lot of tiny delays.
You should be fine. 200 milliseconds is a loooooong time.
For comparison, when I maintained WoW addons, I had a simple database addon. A full-text search of ~130,000 records, each one being about 12 characters long (so, a little longer than average word length), took about 120 milliseconds in the worst case. (Sorting the results took another 200 milliseconds, but that's a different problem.)
Java is faster than Lua, especially with a JIT compiler, and I doubt you'll ever be working with that many records anyway. Plus, it sounds like your algorithm is not a full-text search, it's a longest-prefix search. Muuuuuch faster with the right data structure. For that database, I also built an autocomplete tool that would look at the characters that had been typed, then determine which records started with those characters. With pre-sorted data in a big ol' array, this is a binary search, and took under 1 millisecond for all ~130,000 records. I would have used a trie, but Lua's data structures are not really flexible enough to implement one efficiently.
My Java certification exam is tomorrow at 2pm. I'm getting really nervous. I'm doing better on the practice exams than I was last time, but I don't know if that's because I am remembering the answers or if I'm finally starting to get it.
Anyone have any good resources for writing your own language? Or at least designing one? I'd like to read something cool like that.
Here's how most people start:
* Design your syntax.
* Design a way to parse it and read it in a meaningful way.
* Translate it into another language.
* Compile it with a compiler.
I think that's how C++ got started, it was basically a giant C metaprocessor of some variety. Can you make a dinky language and convert your code to C++? Expand it? Add more? There you go, you're on your way.
Having the compiler spit out it's own binary isn't nearly as important as grammar and syntax and use.
not a doctor, not a lawyer, examples I use may not be fully researched so don't take out of context plz, don't @ me
My copy of Programming Pearls arrived yesterday. Should be a good read. I read the first column (as he likes to call them for some reason, it's just a specific short section or case study) last night. It really drove home just how different what I do all day, writing web apps and whatnot in interpreted languages, from doing much closer to the metal stuff. Using a bit array to sort a list of 10 million 7 digit numbers in just over 1MB of ram because of a memory constraint? Crazy. Dead simple and obvious (for this specific case) once you see it, but not something that ever would have crossed my mind were I presented with that problem before last night.
The picture of a Trie on wikipedia makes it look very useful, thanks. Either that or Deque should do it.
I know 0.2 seconds is a lot, but if I'm tracking every key and every word a user types, and trying to predict the word they're typing and the sentence or sentence fragment, I've got a lot of overhead attached to each keypress in the program, and I need to keep it quick. If I just stored every word the user entered it wouldn't be long before it's trying to work out which of the 8000 words starting with T has been used the most, and if there's even a tiny delay on keypress then there are going to be an awful lot of tiny delays.
Even if you're writing the most speed-efficient search code in the world, you shouldn't actually be doing the searching in the GUI thread between keypresses. Keep a counter in the main thread for a search ID, and one for the last search updated in the GUI. Every time you kick off a search, increment the search ID counter and pass that in as one of the arguments. Every time a search thread finishes, update the GUI thread with the results if the searchID is greater than the last updated search.
Used this exact approach for some slow autocomplete code and it worked perfectly to take care of the keypress delays. A later pass made it much faster anyway, but the users didn't really notice because it didn't feel slow to them.
Posts
First time I've needed to use beautiful soup, but it almost certainly saved me a lot of hassle. I'm supposed to write a deal which will log into an ftp, download an xml file, parse it, pull some data out, and then do stuff with it. Nothing too crazy, right? I download the xml to see what the format is as I have no documentation on that. The first thing my I catch are <font> tags. This is actually a broken html document which has <html>, <meta>, <title> (not inside a <head>), <body>, some old school html 4 <center> and <font> tags, a couple <br> and then suddenly an xml doc starts with no dtd. I get to the bottom of it... there's a closing xml tag with no matching opening tag and then no closing tags for the html.
beatuiful soup managed to make sense of it and let me pull out the not-broken part of the xml and get to work anyway.
Oh what's that, adding your software to the firewall exception list caused it to work. You don't say.
Them: "I think your program is making other programs hang up."
Me: "It shouldn't even effect it. It's hanging up for me too."
Me: "I attached a debugger and it is hanging up inside *enterprise security thingie that intercepts windows api calls." It seems to be caused by a race condition."
Them: "I don't know what a 'debugger' is or a 'race condition' and we can't uninstall *enterprise security thingie that intercepts windows api calls" because it is company policy to have it installed. Since I can't understand or test your explanation I'm going to just rebuild the users' computers and have meetings for weeks. Thanks though."
I think I need to find a new job. We currently have at least three levels of "enterprise security thingie" from various companies and no one thinks that is why absolutely everyone's computer runs slow and hangs randomly.
I told the guy, "You are honestly lucky I can't bill you for my services because this thing that took me 5 minutes to do and you 8 hours, and still unresolved, yesterday should be charged at a premium."
I honestly can't think of a library that makes me happier than Beautiful Soup. It makes a task that would otherwise be horrible and tedious simple and, dare I say it, fun. It is wonderful.
I made a game, it has penguins in it. It's pay what you like on Gumroad.
Currently Ebaying Nothing at all but I might do in the future.
Well, the guys who sell the $500 software aren't going to wine and dine the CTO, fly him to a conference, and take him golfing...
I wish I was joking.
Oh god, you better start running now if it's anything like your OS agnostic work.
Different project completely.
Nintendo ID: Incindium
PSN: IncindiumX
Still have my own desk and can finally start eating again. :P The desks are very, very tiny though.
Yeah, you've stumbled upon the only weakness I've found when dealing with LINQ so far.
Of course, I tend to think that chains of LINQ calls that go more than, say, 4 deep, are a code smell anyway, so I will park them in intermediate variables during initial development. If I have a chain of calls larger than that I actually like to refactor that out into a named method that expresses what the intent of chaining all that together is, just for the sake of readability.
EDIT: Wait, now that I think about it, isn't there a way to load your XDocument with the namespace so you don't have to worry about that? Yeah, this call should help.
Nintendo ID: Incindium
PSN: IncindiumX
If it's not possible I guess I'll just have to work a little more logic into the Vector, and probably tie it all up in a new class that extends a Map, but that's sounding like a complicated solution to my one key -> multiple values problem.
What you describe is a multimap, and the typical implementation of one.
(I'm trying to store most commonly used words, with the first letter as the key, but I want the file to not get stupidly big, and only storing the last 5 or 10 keeps the file small, and also purges uncommon words)
For what you're trying to do, how would that work anyway? How do you decide to evict anything without keep track of a lot more? Plus, storage for say, q x z y v w could be better used expanding other more common letters and so on
What do you need these common words for?
That is a map that stores fixed-length deque objects as its values.
Make a deque class that allocates an object array of size X, you keep an index value to show you where the next insert is and just fill it up as you go. Once you get over X you naturally wrap around, and you have a full circular buffer. The container structure is static so it is about as efficient as you'll get, you just naturally overwrite the oldest stored value as you go.
Would ArrayDeque.push(element) do that circular buffering? Or will I need to implement my element adding specially?
I know 0.2 seconds is a lot, but if I'm tracking every key and every word a user types, and trying to predict the word they're typing and the sentence or sentence fragment, I've got a lot of overhead attached to each keypress in the program, and I need to keep it quick. If I just stored every word the user entered it wouldn't be long before it's trying to work out which of the 8000 words starting with T has been used the most, and if there's even a tiny delay on keypress then there are going to be an awful lot of tiny delays.
For comparison, when I maintained WoW addons, I had a simple database addon. A full-text search of ~130,000 records, each one being about 12 characters long (so, a little longer than average word length), took about 120 milliseconds in the worst case. (Sorting the results took another 200 milliseconds, but that's a different problem.)
Java is faster than Lua, especially with a JIT compiler, and I doubt you'll ever be working with that many records anyway. Plus, it sounds like your algorithm is not a full-text search, it's a longest-prefix search. Muuuuuch faster with the right data structure. For that database, I also built an autocomplete tool that would look at the characters that had been typed, then determine which records started with those characters. With pre-sorted data in a big ol' array, this is a binary search, and took under 1 millisecond for all ~130,000 records. I would have used a trie, but Lua's data structures are not really flexible enough to implement one efficiently.
Here's how most people start:
* Design your syntax.
* Design a way to parse it and read it in a meaningful way.
* Translate it into another language.
* Compile it with a compiler.
I think that's how C++ got started, it was basically a giant C metaprocessor of some variety. Can you make a dinky language and convert your code to C++? Expand it? Add more? There you go, you're on your way.
Having the compiler spit out it's own binary isn't nearly as important as grammar and syntax and use.
things like Java have those kinds of operations well optimized, or at least have ways you can write them to be super fast
Used this exact approach for some slow autocomplete code and it worked perfectly to take care of the keypress delays. A later pass made it much faster anyway, but the users didn't really notice because it didn't feel slow to them.
Slow as beans with a huge list.