The new forums will be named Coin Return (based on the most recent vote)! You can check on the status and timeline of the transition to the new forums here.
Please vote in the Forum Structure Poll. Polling will close at 2PM EST on January 21, 2025.

[Programming] Page overflow to new thread

195969798100

Posts

  • crabjuicecrabjuice Registered User regular
    TelMarine wrote: »
    I find myself confused with the term "stream". Such as "streaming data", "data stream", "stream over data", "stream from a file/database", or any such phrase including "stream" in a computer context. Maybe some here can help clear some of my confusion (paging @Phyphor ).

    My confusion is I'm not sure what's so different about it compared to just having a list that holds data or some data structure you can loop over. What's the difference between looping over a list and reading from a stream? Is the idea that a stream has no known size so you read pieces at a time until you get null or equivalent? However, you can also "stream" over a list with a known size or a file with a known size (such as in Java and Go and probably others). Is reading a file say 4KB at a time considered "streaming"? Is that equivalent to "reading the file"? Or is loading the entire file into a single data structure different than "streaming" it? Is the idea with a array/list that you can immediately access a specific piece of data via index but in a stream you can't since it may not have a known size?

    In Java 8 Stream was added, allowing you to do some functional programming style code. It lets you apply functions to each element in a Stream (of which a Stream can be made from an existing data structure, like a List), but how's that different from looping over that same List with a standard for loop, and just calling functions on each element?

    I think I'm getting caught on definition. A list holds some data of a specific size. A "stream" then is...what?

    This is, I think, a term that's pretty common to be confused over because, afaict, it has no specific technical definition and it gets used in a lot of contexts and is a bit overloaded (because it's an evocative metaphor).

    The definition that I've come to, is that a "stream" is a source of data whose length you don't know in advance and you won't know until you reach the end (and it may not have an end).

    At the low level, IO Streams are a good example of this. When reading a file from disk you don't know how big the file is or when to stop reading the file until you get the EOF signal. Likewise, when reading input from the console, you don't know when to stop reading the characters a user is typing until the user sends a specific signal (like, the enter key). So, these are both called "streams".

    At a higher level of abstraction, a stream would be reading reading from Kafka vs reading from MySQL. A MySQL table has a known size which is fixed during a transaction, so you can do something like Count or Sum and you'll always get a result. However, you can't simply do a count on a Kafka topic because new events are continuously being added, so you'll never known when to stop counting and emit the result. Here (as with the file stream) you need either some kind of EOF signal to tell the counter to stop counting or you need to use something like windowed counts.

    As for Java's Stream API, it's been a long time since I've done anything in the Java ecosystem (so this is just a guess), but I think you should treat it's name as more marketing than being related to streams specifically. My guess is it's called "Stream" because it's a catchy name and because the library can handle both non-deterministic streams and deterministic collections (like Array and List).

  • This content has been removed.

  • KakodaimonosKakodaimonos Code fondler Helping the 1% get richerRegistered User regular
  • admanbadmanb unionize your workplace Seattle, WARegistered User regular
    Anyone else have a scrum master who shows the sprint burn down chart every day at the stand up? Is this considered normal? I've never encountered this and it seems absolutely insane.

    what is "normal" what is "insane" and what is "entirely unsurprising" is a venn diagram that's just a circle

  • gavindelgavindel The reason all your software is brokenRegistered User regular
    The sprints are made up and the points don't matter.

    Book - Royal road - Free! Seraphim === TTRPG - Wuxia - Free! Seln Alora
  • NaphtaliNaphtali Hazy + Flow SeaRegistered User regular
    I actually considered applying for a scrum master job last year because I just wanted to get out my current position

    But then I decided having a job where I filled in JIRA tickets and made spreadsheets might actually be worse

    Steam | Nintendo ID: Naphtali | Wish List
  • This content has been removed.

  • hlprmnkyhlprmnky Registered User regular
    Oh man I got Opinions about “agile” as she is played

    If this scrum master has been doing this for more than one sprint, ask to see their moving average of estimates vs actual time to completion for this team. If they don’t have that, tell them your Internet friend called them a fraud. If they do, by some miracle, have that metric tracked, ask them what their top two things are they plan to try changing based on the variance they see, and what their acceptable average deviation from estimate is, and how long they expect it to take to get the team there. Again, failure at any of these gates is an invitation to tell them you have an Internet fellow traveler who yelled “Fraud!” about it.

    Note that I am not trained in the Scrum(tm) methodology nor have I ever worked as a scrum master. I just have a deep in the bones weariness of a tool that presumably works well when used well, being used instead to yell “IS IT DONE YET, NERDS!?” with ever-increasing granularity and stridency.

    _
    Your Ad Here! Reasonable Rates!
  • schussschuss Registered User regular
    Yep. Part of my job is implementing metrics around lots of agile things. No one worth their salt in the space hyperfocuses on burn downs, velocity or that shit. They focus on cycle and lead time, practice rigor, team roadblocks and streamlining planning processes to make the runway easier to understand and more fleshed out dependency wise so you don't get any crazy story or epic churn.

  • EchoEcho ski-bap ba-dapModerator, Administrator admin
    I know why it happens, but it's still always funny.

    https://mastodon.social/@simevidas/109919980697679274
    If you drag an emoji family with a string size of 11 into an input with maxlength=10, one of the children will disappear.

  • FiskebentFiskebent DenmarkRegistered User regular
    The Agile manifesto has some very good points. Unfortunately it was published without any guidance and so we ended up with Scrum and SAFe and what have you, since people desperately needed guidance on how to actually do it.

    We might be at a point now where people have enough experience with Agile to go back to the manifesto and implement what was intended without the frameworks. But I'm afraid that it's too late and Scrum and SAFe are so entrenched now that it's impossible to change.

    steam_sig.png
  • ZekZek Registered User regular
    edited February 2023
    Dang, I hadn't really found a great use case for ChatGPT before, but it's amazingly useful for learning new frameworks. When I have a very specific question about something, Google's results are only useful if my question specifically matches some article or StackOverflow answer. Official documentation is way too technical for a first timer. ChatGPT pretty much has the knowledge from the official documentation though, and can contextualize it however you want. That's a game changer. It's okay if it gets the specific code wrong because I'll have autocomplete for that. I'm annoyed that my job has so much proprietary tech it doesn't know about.

    Zek on
  • LD50LD50 Registered User regular
    My friend is using it to write cover letters for job applications.

  • DisruptedCapitalistDisruptedCapitalist I swear! Registered User regular
    My wife used it to write a polite letter declining additional work when she really wanted to write a sarcastic and bitter screed.

    "Simple, real stupidity beats artificial intelligence every time." -Mustrum Ridcully in Terry Pratchett's Hogfather p. 142 (HarperPrism 1996)
  • DrovekDrovek Registered User regular
    My wife used it to write a polite letter declining additional work when she really wanted to write a sarcastic and bitter screed.

    Nothing beats a good old "lol no"

    steam_sig.png( < . . .
  • EchoEcho ski-bap ba-dapModerator, Administrator admin
    Zek wrote: »
    Dang, I hadn't really found a great use case for ChatGPT before, but it's amazingly useful for learning new frameworks. When I have a very specific question about something, Google's results are only useful if my question specifically matches some article or StackOverflow answer. Official documentation is way too technical for a first timer. ChatGPT pretty much has the knowledge from the official documentation though, and can contextualize it however you want. That's a game changer. It's okay if it gets the specific code wrong because I'll have autocomplete for that. I'm annoyed that my job has so much proprietary tech it doesn't know about.

    Yeah, I've used it for stuff like "How do I do X in Terraform to deploy on AWS?" and it gives me good enough answers. I see what it's trying to do and can fix the errors.

  • LD50LD50 Registered User regular
    edited February 2023
    I should ask it how to vertically center a div in css.

    LD50 on
  • GnomeTankGnomeTank What the what? Portland, OregonRegistered User regular
    The only not-work-related programming I do these days is in C++, inside of Unreal...and let me tell you, C++ has become a real weird language.

    Sagroth wrote: »
    Oh c'mon FyreWulff, no one's gonna pay to visit Uranus.
    Steam: Brainling, XBL / PSN: GnomeTank, NintendoID: Brainling, FF14: Zillius Rosh SFV: Brainling
  • djmitchelladjmitchella Registered User regular
    Ugh, just lost a morning trying to work out why, for some reason, my server's endpoint was getting hit twice even though my code only sends one fetch() request, and the browser tab only shows one request being made. (it's not the CORS preflight request, either, and this doesn't happen if I make a GET request from the address bar, or from Postman). Googling got me an awful lot of hits that were not the right answer and were 'my webpage was refreshing unexpectedly' or 'react is complicated' or 'my webpack config was broken', or 'I don't understand CORS', etc.

    The right answer, it turns out, is buried in a 'MAY' clause of an RFC. This particular endpoint tries to do something that might, for various annoying reasons, not ever complete, so it times out instead of waiting forever. So I was returning 408 'request timed out', which seemed like the right thing.

    But, buried at the end of the relevant bit of the RFC, it says "If the client has an outstanding request in transit, the client MAY repeat that request on a new connection.". And that's exactly what was happening, and that's why the browser said that my calls took twice as long to fail as I was expecting, and why my endpoint was getting hit twice for a single GET request.

    Fix: return 504 gateway timeout instead -- in practise, it's something elsewhere that we were waiting for that didn't respond, so that was probably the right response in the first place, to be fair. But sheesh, I sure did not want to use up most of friday afternoon and all of this morning tracking down the side effects of picking the wrong sort of "timeout" response code.

  • EchoEcho ski-bap ba-dapModerator, Administrator admin
    Naphtali wrote: »
    I actually considered applying for a scrum master job last year because I just wanted to get out my current position

    But then I decided having a job where I filled in JIRA tickets and made spreadsheets might actually be worse

  • Ear3nd1lEar3nd1l Eärendil the Mariner, father of Elrond Registered User regular
    GnomeTank wrote: »
    The only not-work-related programming I do these days is in C++, inside of Unreal...and let me tell you, C++ has become a real weird language.

    I've decided that I no longer do non-work related programming. After nearly 30 years of doing this, I need a new hobby.

  • dporowskidporowski Registered User regular
    edited February 2023
    Ear3nd1l wrote: »
    GnomeTank wrote: »
    The only not-work-related programming I do these days is in C++, inside of Unreal...and let me tell you, C++ has become a real weird language.

    I've decided that I no longer do non-work related programming. After nearly 30 years of doing this, I need a new hobby.

    Q: "Are you doing Advent of Code this year?!?!"

    A: "Are they paying me?"

    dporowski on
  • SaerisSaeris Borb Enthusiast flapflapflapflapRegistered User regular
    Anyone familiar with Numpy?

    I've got an array where each element has a count and a score. I want to index the array using some other array J of indices, then increment each element's count by 1 and score by S[n], where S is an array of the same length as J.

    In code:
    import numpy as np
    d = np.dtype([('count', np.uint32), ('score', np.float32)])
    a = np.zeros(5, dtype=d)
    idx = [2, 4]
    scores = [3.0, 6.0]
    a[idx] += (1, scores)
    

    I was hoping Numpy would recognize that a) the value 1 can be broadcast to match the shape of a[idx], and b) the shape of scores is the same as the shape of a[idx]. But instead it raises an exception:
    ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
    

    I can just increment the two fields separately:
    a['count'][idx] += 1
    a['score'][idx] += scores
    

    But I feel like that's gotta have worse cache behaviour, hitting the indexed elements twice. How can I access each element just once?

    borb_sig.png
  • schussschuss Registered User regular
    Ear3nd1l wrote: »
    GnomeTank wrote: »
    The only not-work-related programming I do these days is in C++, inside of Unreal...and let me tell you, C++ has become a real weird language.

    I've decided that I no longer do non-work related programming. After nearly 30 years of doing this, I need a new hobby.

    I saw in the Lego thread they make a raspberry pi that integrates with technic stuff. You know, if you're into that sort of thing.
    (I used some the other day to simulate stat arrays from the new paizo playtest dice rules)

  • crabjuicecrabjuice Registered User regular
    Saeris wrote: »
    Anyone familiar with Numpy?

    I've got an array where each element has a count and a score. I want to index the array using some other array J of indices, then increment each element's count by 1 and score by S[n], where S is an array of the same length as J.

    In code:
    import numpy as np
    d = np.dtype([('count', np.uint32), ('score', np.float32)])
    a = np.zeros(5, dtype=d)
    idx = [2, 4]
    scores = [3.0, 6.0]
    a[idx] += (1, scores)
    

    I was hoping Numpy would recognize that a) the value 1 can be broadcast to match the shape of a[idx], and b) the shape of scores is the same as the shape of a[idx]. But instead it raises an exception:
    ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
    

    I can just increment the two fields separately:
    a['count'][idx] += 1
    a['score'][idx] += scores
    

    But I feel like that's gotta have worse cache behaviour, hitting the indexed elements twice. How can I access each element just once?

    I don't know Numpy, but I think the cache performance of the second example will be the same as the first example (assuming there's a way to do what you want), because the two will probably compile to the same sequence of instructions that are executed by the CPU.

    The cache behavior depends more on the location of the elements to which you're adding: if they're in a contiguous block of memory or all in the same cache line (or contiguous cache lines) then the first time you read one of the elements they'll all be loaded into the L1 cache (prefetching would pull in adjacent cache lines). If they're not next to each other in memory then you'll have a cache miss for each element that you read no matter which code design you use (unless you or Numpy manually insert prefetch instructions into the code before you perform the addition, this would cause the elements to be fetched from RAM concurrent with you executing the first arithmetic operation).

    By "access each element just once" I'm guessing you mean you'd like to use a single CPU instruction to perform all the additions for the elements at '[idx]'? Put another way: one instruction reads all the elements at '[idx]', a second instruction adds '(1, scores)', and then a third writes the block of elements back to memory. You'll only be able to get something like that if Numpy uses SIMD instructions behind the scenes. If it doesn't, then it's going to individually read each number from memory, perform the addition, then write the number back to memory. And again, this case depends entirely on how 'a' is laid out in memory.

    (SIMD is Single Instruction Multi Data, this lets you do things like add to vectors of the same length together in a single CPU Op. There's also MIMD (Multi Instruction Multi Data), but I don't know if x86 has those).

    TLDR: Cache performance will almost certainly be the same for both versions of you're code, so don't worry about it and just focus on which version is most readable and easiest to get working.

  • hlprmnkyhlprmnky Registered User regular
    Ear3nd1l wrote: »
    GnomeTank wrote: »
    The only not-work-related programming I do these days is in C++, inside of Unreal...and let me tell you, C++ has become a real weird language.

    I've decided that I no longer do non-work related programming. After nearly 30 years of doing this, I need a new hobby.

    Good old software-engineer-to-woodworker pipeline still putting up numbers. “Woodworker” in this case being a term that can expand to mean fly fishing, or cooking, or music making, or basically any activity where the things, once done, stay accomplished, and nothing involved can ever need an “update” to “dependencies”. I, myself, have taken up sailing on a little boat from 1985 that has one car battery to power the lights, and a depth finder/knot meter with an LCD screen. I can, to my mild displeasure, still get cell service out on the lake but I try hard not to remember that I have it.

    _
    Your Ad Here! Reasonable Rates!
  • Ear3nd1lEar3nd1l Eärendil the Mariner, father of Elrond Registered User regular
    I've taken up learning Blender. It's been very cathartic.

  • TelMarineTelMarine Registered User regular
    hlprmnky wrote: »
    Ear3nd1l wrote: »
    GnomeTank wrote: »
    The only not-work-related programming I do these days is in C++, inside of Unreal...and let me tell you, C++ has become a real weird language.

    I've decided that I no longer do non-work related programming. After nearly 30 years of doing this, I need a new hobby.

    Good old software-engineer-to-woodworker pipeline still putting up numbers. “Woodworker” in this case being a term that can expand to mean fly fishing, or cooking, or music making, or basically any activity where the things, once done, stay accomplished, and nothing involved can ever need an “update” to “dependencies”. I, myself, have taken up sailing on a little boat from 1985 that has one car battery to power the lights, and a depth finder/knot meter with an LCD screen. I can, to my mild displeasure, still get cell service out on the lake but I try hard not to remember that I have it.

    Everything requires maintenance. Don't boats especially need a bit of upkeep and cleaning?

    3ds: 4983-4935-4575
  • SoggybiscuitSoggybiscuit Tandem Electrostatic Accelerator Registered User regular
    Wasn't quite sure what thread to ask this in (this or sysadmin thread), but I need a good data visualization software similar to grafana that can do some non-time series data stuff. It's not anything mission critical, but I'd like to monitor the data coming in for my thesis experiments and I'm working on a setup that uses influxdb to do just that so I'm not bugging my fellow graduate students while they run shifts for me. Problem is, I do need to make a couple of different 2D histograms that are not just time-series based to check certain correlations in the data*. I could download the data remotely, but I have to VPN in to campus, ssh into one machine, ssh tunnel from that machine to a different machine behind a firewall to get it and I have zero desire to do that especially when I'll be going in daily to collect my data in person anyways. I just want to check up a couple of times a day when I'm not there to make sure everything is going okay. I'm hoping someone here knows of a different (FREE, I'm a graduate student) solution.

    * I took 10 hours of data recently covering a energy range of interest that is significantly less usable because I set a threshold on an ADC too high, so I would like to avoid this in the future.

    Steam - Synthetic Violence | XBOX Live - Cannonfuse | PSN - CastleBravo | Twitch - SoggybiscuitPA
  • hlprmnkyhlprmnky Registered User regular
    TelMarine wrote: »
    hlprmnky wrote: »
    Ear3nd1l wrote: »
    GnomeTank wrote: »
    The only not-work-related programming I do these days is in C++, inside of Unreal...and let me tell you, C++ has become a real weird language.

    I've decided that I no longer do non-work related programming. After nearly 30 years of doing this, I need a new hobby.

    Good old software-engineer-to-woodworker pipeline still putting up numbers. “Woodworker” in this case being a term that can expand to mean fly fishing, or cooking, or music making, or basically any activity where the things, once done, stay accomplished, and nothing involved can ever need an “update” to “dependencies”. I, myself, have taken up sailing on a little boat from 1985 that has one car battery to power the lights, and a depth finder/knot meter with an LCD screen. I can, to my mild displeasure, still get cell service out on the lake but I try hard not to remember that I have it.

    Everything requires maintenance. Don't boats especially need a bit of upkeep and cleaning?

    Oh certainly they do, but to stretch the analogy to (or past!) its breaking point, there is a world of difference between “this line looks a little frayed, I should swap it end for end and check these blocks and guides, make sure nothing is dragging on it” and “due to decisions made elsewhere to serve the needs of others, only boats with YELLOW sails are now able to engage the wind. Please install or fabricate YELLOW sails to continue using your boat”, which is the not-woodworking nature of software development.

    _
    Your Ad Here! Reasonable Rates!
  • admanbadmanb unionize your workplace Seattle, WARegistered User regular
    hlprmnky wrote: »
    TelMarine wrote: »
    hlprmnky wrote: »
    Ear3nd1l wrote: »
    GnomeTank wrote: »
    The only not-work-related programming I do these days is in C++, inside of Unreal...and let me tell you, C++ has become a real weird language.

    I've decided that I no longer do non-work related programming. After nearly 30 years of doing this, I need a new hobby.

    Good old software-engineer-to-woodworker pipeline still putting up numbers. “Woodworker” in this case being a term that can expand to mean fly fishing, or cooking, or music making, or basically any activity where the things, once done, stay accomplished, and nothing involved can ever need an “update” to “dependencies”. I, myself, have taken up sailing on a little boat from 1985 that has one car battery to power the lights, and a depth finder/knot meter with an LCD screen. I can, to my mild displeasure, still get cell service out on the lake but I try hard not to remember that I have it.

    Everything requires maintenance. Don't boats especially need a bit of upkeep and cleaning?

    Oh certainly they do, but to stretch the analogy to (or past!) its breaking point, there is a world of difference between “this line looks a little frayed, I should swap it end for end and check these blocks and guides, make sure nothing is dragging on it” and “due to decisions made elsewhere to serve the needs of others, only boats with YELLOW sails are now able to engage the wind. Please install or fabricate YELLOW sails to continue using your boat”, which is the not-woodworking nature of software development.

    I don't want to harp on this but "this very specific part on my boat broke and to identify it we have to stare at a 30-year-old rusted over serial number and then figure out if that manufacturer (a) still exists and (b) still makes the part or an acceptable replacement, because if we can't I have to tear out the entire plumbing system" is specifically a problem that boat owners have to deal with.

    source: partner worked at a marine supply store for 10ish years.

  • hlprmnkyhlprmnky Registered User regular
    I don’t disagree with that statement at all; it’s definitely possible to have a boat that has frustrating and complex maintenance issues. It’s even possible to have a boat with onboard wifi responsible for collating data streams from disparate sensors with proprietary data transfer formats for display on a weatherproof touchscreen with a repeater to the chart table nav workstation, to say nothing of complex plumbing arrangements with electric flush toilets, water heaters, inverters for AC power to drive your kitchen appliances underway, and on and on.

    But on my boat, there isn’t even an inboard diesel. There’s a forty year hold Honda 9.9HP outboard for which I have the shop manual, and a 12V breaker panel with six circuits labeled things like “running lights” and “depth sounder”. I do a fair amount of maintenance and even tinkering to improve things, but the very edge of the “inscrutable bullshit” envelope for systems under my responsibility is servicing a manual sheet winch or maybe giving a carburetor a bath in some Gum-Out. I do think we have reached the end of where the analogy serves the topic, though. Tonight I will dream an uncomfortable dream of plumbing maintenance in tight fiberglass spaces.

    _
    Your Ad Here! Reasonable Rates!
  • gavindelgavindel The reason all your software is brokenRegistered User regular
    why are we doing this?
    Its the requirement.
    Why is it the requirement?
    The PM set the requirement.
    Which PM?
    He's not with the company anymore.
    So why are we doing this?
    Because its the requirement.

    Book - Royal road - Free! Seraphim === TTRPG - Wuxia - Free! Seln Alora
  • CarpyCarpy Registered User regular
    edited March 2023
    Love starting out the week by screwing up my branches badly enough that it's easier to just reclone the whole repo instead of fixing my bullshit

    Carpy on
  • EchoEcho ski-bap ba-dapModerator, Administrator admin
    Carpy wrote: »
    Love starting out the week by screwing up my branches badly enough that it's easier to just reclone the whole repo instead of fixing my bullshit

    I confess I've had a few instances of "how many senior engineers does it take to unfuck this weird Git situation we ended up in somehow?", yes.

  • AkimboEGAkimboEG Mr. Fancypants Wears very fine pants indeedRegistered User regular
    edited March 2023
    A really neat one I had recently: When working on a new repo, due to merging shenanigans git was expecting both a README.md and a readme.md in my current commit, but I'm on a Windows machine, where filenames aren't case sensitive. So I can't pull out of the current commit, because whatever I do, there's going to be at least one uncommitted change. Had to `git reset --hard`.

    AkimboEG on
    Give me a kiss to build a dream on; And my imagination will thrive upon that kiss; Sweetheart, I ask no more than this; A kiss to build a dream on
  • djmitchelladjmitchella Registered User regular
    My fix for all git issues is:

    1. rename existing repo to "repo.backup".
    2. make new clone of repo
    3. manually copy-paste changes over between files

    I'm sure there's better ways, but I _trust_ this approach to not mess me up in confusing ways, whereas every time I've tried to do things properly I just make it worse.

  • DrovekDrovek Registered User regular
    My fix for all git issues is:

    1. rename existing repo to "repo.backup".
    2. make new clone of repo
    3. manually copy-paste changes over between files

    I'm sure there's better ways, but I _trust_ this approach to not mess me up in confusing ways, whereas every time I've tried to do things properly I just make it worse.

    I have "senior" somewhere randomly in my title... this is still my go-to "unfuck git" approach.

    steam_sig.png( < . . .
  • admanbadmanb unionize your workplace Seattle, WARegistered User regular
    I don't think I've fucked up git in a way that couldn't be fixed with git reset --hard in years. wtf are you all doing?

  • EchoEcho ski-bap ba-dapModerator, Administrator admin
    Man, if I knew, I would stop doing it.

This discussion has been closed.