Anyone got any suggestions for software/libraries for doing high dimensional data clustering. Looking at vectors with around ~10,000 dimensions almost all of which will be binary 1,0 but a few (like less than a dozen) with continuous data
PCA to reduce that down if it's any kind of sparse
I think lasso is strong choice for very wide data. At least, it gets thrown around in a compbio setting a lot.
Oh yeah, this shit is getting PCA'd as it is helluva sparse.
I've been trying scikitlearn and it seems to have everything i need.
GnomeTankWhat the what?Portland, OregonRegistered Userregular
Trying to wrap my head around React + Redux. Can someone explain to me what this is meant to buy me over traditional MVC? Or even Angular's component based model? It all seems needlessly complex, and I hate to write it off as just the trendy thing...there has to be a reason people are excited by the combo.
I am really liking RxJS though. Seems like a natural and needed extension to the Promise model. Especially the ability to map/reduce/slice the observable streams.
Trying to wrap my head around React + Redux. Can someone explain to me what this is meant to buy me over traditional MVC? Or even Angular's component based model? It all seems needlessly complex, and I hate to write it off as just the trendy thing...there has to be a reason people are excited by the combo.
I am really liking RxJS though. Seems like a natural and needed extension to the Promise model. Especially the ability to map/reduce/slice the observable streams.
Trying to wrap my head around React + Redux. Can someone explain to me what this is meant to buy me over traditional MVC? Or even Angular's component based model? It all seems needlessly complex, and I hate to write it off as just the trendy thing...there has to be a reason people are excited by the combo.
Easy to reason about data-flow and state rules.Re-frame's documentation explains a lot of what redux is trying to do(while also pre-dating).
Edit: I was answering only about redux, were you asking about React vs NG too? I thought you were already working with React?
First, React is great, and it’s a great choice for building a web app these days (better than Angular which is a piece of shit IMO). The big thing to grok is the concept of a one way data flow. Data trickles down your component trees in the form of props, it never flows upstream. A nice thing is that your components can be defined completely within one file too. This keeps rendering logic close to the state logic, which is right where you want it to be, and it leads to TRUE encapsulation of concerns.
Redux: Most of the time, you don’t need Redux IMO. Every component in React maintains it’s own local state. That state cannot be shared with other components. Redux is basically a large global state for your entire application. If there is state that you wish to share throughout your application (for example, a logged in User) the best place to keep that state is in a Redux store. This gives you a single source of truth for your global state. Redux also follows a pattern of passing objects called “actions” to reducers. The reducer uses the action object to modify the global state. This is necessary because global state is immutable, and convenient because you can maintain a log of all the actions as they modify your global state.
What does it get you over MVC?
Compared to MVC, React makes it trivial to create applications that have a complex series of interactions. As an application grows, an MVC architecture results in an explosion of relations that make it very hard to reason about your code.
Listen, I used to build a lot of iOS apps and that was all done in an MVC fashion, after moving to React and React-Native though I can see now how cumbersome it would be to do some of the things I do now with React.
All opinions are my own!
PS. React is NOT complex at all, a lot of beginners to it say this but it's an illusion. The reason it seems complex is because it's unlike any other sort of web development paradigms. However, if you've built applications that run anywhere other than a browser, you'll find React is closer to real application development vs web app development.
vimmer on
0
Options
GnomeTankWhat the what?Portland, OregonRegistered Userregular
Trying to wrap my head around React + Redux. Can someone explain to me what this is meant to buy me over traditional MVC? Or even Angular's component based model? It all seems needlessly complex, and I hate to write it off as just the trendy thing...there has to be a reason people are excited by the combo.
Easy to reason about data-flow and state rules.Re-frame's documentation explains a lot of what redux is trying to do(while also pre-dating).
Edit: I was answering only about redux, were you asking about React vs NG too? I thought you were already working with React?
No, I am mostly an Angular developer. I am learning React + Redux as personal enrichment.
Trying to wrap my head around React + Redux. Can someone explain to me what this is meant to buy me over traditional MVC? Or even Angular's component based model? It all seems needlessly complex, and I hate to write it off as just the trendy thing...there has to be a reason people are excited by the combo.
I am really liking RxJS though. Seems like a natural and needed extension to the Promise model. Especially the ability to map/reduce/slice the observable streams.
So Alistair's take on why React (and Redux) are the bomb. TL;DR React + Redux allows you to produce simple things that are easy to test.
First, whilst I think you know this, I just want to be clear that React and Redux are separate technologies. You could happily write a React app without ever using Redux and you can use Redux for data routing without using React for the View Layer.
Second, I am a strong believer that two way data binding is the devil. This is pretty key to how React works. React is a reaction (ho ho) against two way data binding. React is a View layer that lets/makes you write functional views with one way data flow. Views are generated by a function taking some passed in values and some local state and generating the view markup. The markup is completely local to the component generating it - there are no weird html template files to find in the code base nor auto-magic linked models and controllers. React is explicit which means, yes there is boiler plate but everything is very straightforward. A React component is a single class, that class has everything in it that you need to understand how the View is generated. Because views are now functional it is very easy to unit test your views React is simple, that is it's strength.
Redux is simpler. I try not to overuse the word genius but Redux is genius. It is the genius of total simplicity. The state of your application is held in a single, immutable data structure. Changes to you program state happen through your React components generating events which are listened to by the Redux Store - which holds the single, immutable data structure. Any changes to the state of your program result in an entirely new data structure being generated and passed to the React view layer. Redux does result in some boile plate but it is simple, understandable scaffolding that makes clear the direction of data in your program. The power of Redux is two fold - once it again it is trivially easy to unit test your Store, every single reducer can be unit tested and then the combinations and so forth, the second is almost magical. given an initial state and a recorded list of actions you can recreate your App state at any point in it's lifetime, you simply apply each action in turn to generate a new state and your React App 'snaps' into place as it's a set of functional views over the data.
It's entirely possible to write a React app with just props, local state and change handlers but once you have something of size the extra effort to integrate Redux greatly simplifies the chains of connections between your components making it easier to move things around as all the data flow is now decoupled from the exact hierarchy of your components
If nothing else I'd seriously recommend watching the Getting Started With Redux tutorial, it's a master piece of teaching. I quite like https://facebook.github.io/react/docs/thinking-in-react.html as well for getting you in the mood for the React way.
First, React is great, and it’s a great choice for building a web app these days (better than Angular which is a piece of shit IMO). The big thing to grok is the concept of a one way data flow. Data trickles down your component trees in the form of props, it never flows upstream. A nice thing is that your components can be defined completely within one file too. This keeps rendering logic close to the state logic, which is right where you want it to be, and it leads to TRUE encapsulation of concerns.
Redux: Most of the time, you don’t need Redux IMO. Every component in React maintains it’s own local state. That state cannot be shared with other components. Redux is basically a large global state for your entire application. If there is state that you wish to share throughout your application (for example, a logged in User) the best place to keep that state is in a Redux store. This gives you a single source of truth for your global state. Redux also follows a pattern of passing objects called “actions” to reducers. The reducer uses the action object to modify the global state. This is necessary because global state is immutable, and convenient because you can maintain a log of all the actions as they modify your global state.
What does it get you over MVC?
Compared to MVC, React makes it trivial to create applications that have a complex series of interactions. As an application grows, an MVC architecture results in an explosion of relations that make it very hard to reason about your code.
Listen, I used to build a lot of iOS apps and that was all done in an MVC fashion, after moving to React and React-Native though I can see now how cumbersome it would be to do some of the things I do now with React.
All opinions are my own!
PS. React is NOT complex at all, a lot of beginners to it say this but it's an illusion. The reason it seems complex is because it's unlike any other sort of web development paradigms. However, if you've built applications that run anywhere other than a browser, you'll find React is closer to real application development vs web app development.
I come from a Windows/WPF background, and Angular feels much more like that world to me than React does...but it's quite possible I just don't know React well enough yet.
Anyone got any suggestions for software/libraries for doing high dimensional data clustering. Looking at vectors with around ~10,000 dimensions almost all of which will be binary 1,0 but a few (like less than a dozen) with continuous data
PCA to reduce that down if it's any kind of sparse
I think lasso is strong choice for very wide data. At least, it gets thrown around in a compbio setting a lot.
Oh yeah, this shit is getting PCA'd as it is helluva sparse.
I've been trying scikitlearn and it seems to have everything i need.
PCA is actually not always great with very sparse data... I'm trying to remember what I used a couple years ago on something similar, I'll try to find it
For what it's worth, Angular comes in two versions, Angular 1 and "Angular 2", (which will hit v4 any moment now). They are very much not the same -- we're currently doing new development in Angular 2, and Angular 2 is just fine compared to other modern frameworks -- the existing big codebase we have to maintain is angular 1, and it is a lot more painful. (partly because it needs some cleanup because of history / age / etc, but also because angular 1 just doesn't feel right).
For what it's worth, Angular comes in two versions, Angular 1 and "Angular 2", (which will hit v4 any moment now). They are very much not the same -- we're currently doing new development in Angular 2, and Angular 2 is just fine compared to other modern frameworks -- the existing big codebase we have to maintain is angular 1, and it is a lot more painful. (partly because it needs some cleanup because of history / age / etc, but also because angular 1 just doesn't feel right).
Since I spoke out harshly about Angular, I should also clarify most of that is directed toward Angular 1. I don't have experience with Angular 2, and if I have aversion to it it might only be because the Angular brand left a bad taste in my mouth.
Yes, Angular 1 though, I feel is garbage, and I've never met an organization that did not express regrets about sticking to it. The best description I've heard of Angular 1 is "bandages over self inflicted wounds".
I don't know what Angular 2 looks like, but I wish anyone who chooses to work with it the best of luck, and I hope you're commitment to learning the framework pays off in the future for you and your loved ones.
0
Options
GnomeTankWhat the what?Portland, OregonRegistered Userregular
Angular 1 is only bad when people try and bend it to their will, rather than just doing things the "Angular way". It's a very opinionated framework, but if you agree with those opinions and just go with it, it's completely workable. I've written some relatively large, very stable, apps in Angular 1 with no major complaints.
I do most of my development these days in Angular 2, which is definitely a better framework.
In my view, Angular 1 is for sending data to a client-side web page and having it perform data-sensitive HTML rendering. End of list.
If you ONLY use it for this and you keep your architecture simple, it's a completely serviceable framework
When you start "developing in angular" that's right about where you fucked up
I'm probably putting words into the Angular 1 developers mouths, but after working in it for two years and managing a team of angular devs, I never got the impression they meant the framework to be the foundation for functionality-rich statically-served thin clients on the web. More like "a thing that lets you put ASP:Repeater workload on the browser instead of the server"
But now entire websites are built basically without any server at all, just a cloud API in a virtualbox somewhere and a JS client that consumes it.. Angular 1 is ill-equipped
Either way, all new web client dev is a trash fire now anyway. I'll be pumping out server rendered pages w/ CSS frameworks and web components till the day i die
god bless never having to have more than 100 concurrent users
0
Options
GnomeTankWhat the what?Portland, OregonRegistered Userregular
Eh, I think the problem is that a lot of people never really learned how to use Angular 1's deeper compositing tools because, frankly, they're obtuse as fuck. You could build completely competent component based architectures using directives and services, but it was not readily apparent how and the Angular devs themselves were very hand wavy about it all. That's where Angular 2 cleans things up. It makes the component a first class construct and the basis for building applications in a much less obtuse manner.
I am definitely in the "cloud API in a container running in some EC2 instance somewhere" camp.
Hmm. Can anyone explain this golang notation to me?
primes := [6]int{2, 3, 5, 7, 11, 13}
var s []int = primes[1:4]
We make an array of prime numbers, then a slice referencing that array. Slice starts at index 1, and then there's what I thought was either the length from the starting index, or the index of the end of the slice.
The slice is [3, 5, 7]. Does the syntax go "from index 1 to the fourth item"? Feels pretty counter-intuitive.
edit:
This slices up to (but excluding) s[5].
OK, so it goes up to but not including the number. Man that feels weird.
Hmm. Can anyone explain this golang notation to me?
primes := [6]int{2, 3, 5, 7, 11, 13}
var s []int = primes[1:4]
We make an array of prime numbers, then a slice referencing that array. Slice starts at index 1, and then there's what I thought was either the length from the starting index, or the index of the end of the slice.
The slice is [3, 5, 7]. Does the syntax go "from index 1 to the fourth item"? Feels pretty counter-intuitive.
edit:
This slices up to (but excluding) s[5].
OK, so it goes up to but not including the number. Man that feels weird.
Sorry, I'm having a hard time writing a reply, since I just threw up all over my screen.
Yep! Still doing a lot of learning, want to get a firmer grasp of Go before I look at the codebase. Had a quick look and it felt like I had no idea what was native to Go, what was external libraries and what came from other parts of the codebase.
Yep! Still doing a lot of learning, want to get a firmer grasp of Go before I look at the codebase. Had a quick look and it felt like I had no idea what was native to Go, what was external libraries and what came from other parts of the codebase.
Good to hear! Hope it's a fun job.
Spend some time and get comfortable with your environment as well. Depending on what editor you are using, something like gocode is invaluable.
You can also slice arrays by doing array[2,4] but that actually means something different: "starting at index 2 grab 4 items", and I don't think I've seen it used.
You can also slice arrays by doing array[2,4] but that actually means something different: "starting at index 2 grab 4 items", and I don't think I've seen it used.
I may have misunderstood what you are saying. Golang syntax is a match for ..., I thought you are forgetting about it and thinking of .. only. My bad.
Maybe I'll use this as an excuse to get better at vim, but that'll wait until I have a firm grasp of go, don't want to struggle with the editor at the same time.
Maybe I'll use this as an excuse to get better at vim, but that'll wait until I have a firm grasp of go, don't want to struggle with the editor at the same time.
I am not a programmer but I lurk here sometimes. There is a difference in '..' and '...'? That seems crazy.
It's not as crazy as it sounds. A range in ruby is a representation of a range of values: 1..10 means "every number between one and ten, including ten", and 1...10 means "every number from one up to but not including ten". You really need both versions because 1...10 is not the same thing as 1..9 (9.5 is higher than 9 but less than 10, for example).
I am not a programmer but I lurk here sometimes. There is a difference in '..' and '...'? That seems crazy.
It's not as crazy as it sounds. A range in ruby is a representation of a range of values: 1..10 means "every number between one and ten, including ten", and 1...10 means "every number from one up to but not including ten". You really need both versions because 1...10 is not the same thing as 1..9 (9.5 is higher than 9 but less than 10, for example).
I get the functionality, but it seems like it would be so easy to make a mistake.
Posts
http://scikit-learn.org/stable/tutorial/machine_learning_map/
If you haven't seen that already
https://docs.google.com/spreadsheets/d/1OLcAGbXhWIVcl5IziVpG0eKFJS3xi_Sac9kYMkRFvD8/edit?usp=sharing
I am really liking RxJS though. Seems like a natural and needed extension to the Promise model. Especially the ability to map/reduce/slice the observable streams.
Give me 30 mins and I will give you my take.
Tldr. React is great
I made a game, it has penguins in it. It's pay what you like on Gumroad.
Currently Ebaying Nothing at all but I might do in the future.
Easy to reason about data-flow and state rules.Re-frame's documentation explains a lot of what redux is trying to do(while also pre-dating).
Edit: I was answering only about redux, were you asking about React vs NG too? I thought you were already working with React?
First, React is great, and it’s a great choice for building a web app these days (better than Angular which is a piece of shit IMO). The big thing to grok is the concept of a one way data flow. Data trickles down your component trees in the form of props, it never flows upstream. A nice thing is that your components can be defined completely within one file too. This keeps rendering logic close to the state logic, which is right where you want it to be, and it leads to TRUE encapsulation of concerns.
Redux: Most of the time, you don’t need Redux IMO. Every component in React maintains it’s own local state. That state cannot be shared with other components. Redux is basically a large global state for your entire application. If there is state that you wish to share throughout your application (for example, a logged in User) the best place to keep that state is in a Redux store. This gives you a single source of truth for your global state. Redux also follows a pattern of passing objects called “actions” to reducers. The reducer uses the action object to modify the global state. This is necessary because global state is immutable, and convenient because you can maintain a log of all the actions as they modify your global state.
What does it get you over MVC?
Compared to MVC, React makes it trivial to create applications that have a complex series of interactions. As an application grows, an MVC architecture results in an explosion of relations that make it very hard to reason about your code.
Listen, I used to build a lot of iOS apps and that was all done in an MVC fashion, after moving to React and React-Native though I can see now how cumbersome it would be to do some of the things I do now with React.
All opinions are my own!
PS. React is NOT complex at all, a lot of beginners to it say this but it's an illusion. The reason it seems complex is because it's unlike any other sort of web development paradigms. However, if you've built applications that run anywhere other than a browser, you'll find React is closer to real application development vs web app development.
No, I am mostly an Angular developer. I am learning React + Redux as personal enrichment.
So Alistair's take on why React (and Redux) are the bomb. TL;DR React + Redux allows you to produce simple things that are easy to test.
First, whilst I think you know this, I just want to be clear that React and Redux are separate technologies. You could happily write a React app without ever using Redux and you can use Redux for data routing without using React for the View Layer.
Second, I am a strong believer that two way data binding is the devil. This is pretty key to how React works. React is a reaction (ho ho) against two way data binding. React is a View layer that lets/makes you write functional views with one way data flow. Views are generated by a function taking some passed in values and some local state and generating the view markup. The markup is completely local to the component generating it - there are no weird html template files to find in the code base nor auto-magic linked models and controllers. React is explicit which means, yes there is boiler plate but everything is very straightforward. A React component is a single class, that class has everything in it that you need to understand how the View is generated. Because views are now functional it is very easy to unit test your views React is simple, that is it's strength.
Redux is simpler. I try not to overuse the word genius but Redux is genius. It is the genius of total simplicity. The state of your application is held in a single, immutable data structure. Changes to you program state happen through your React components generating events which are listened to by the Redux Store - which holds the single, immutable data structure. Any changes to the state of your program result in an entirely new data structure being generated and passed to the React view layer. Redux does result in some boile plate but it is simple, understandable scaffolding that makes clear the direction of data in your program. The power of Redux is two fold - once it again it is trivially easy to unit test your Store, every single reducer can be unit tested and then the combinations and so forth, the second is almost magical. given an initial state and a recorded list of actions you can recreate your App state at any point in it's lifetime, you simply apply each action in turn to generate a new state and your React App 'snaps' into place as it's a set of functional views over the data.
It's simple, it's clever.
If you have not seen them the two EggHead lessons by Redux's creator are amazing. After you have watched the first (which is 2 hours split into 30 beautifully bite sized chunks) you will be able to implement Redux from scratch yourself. Dan amazing at clearly and understandably presenting his technology
https://egghead.io/courses/getting-started-with-redux
https://egghead.io/courses/building-react-applications-with-idiomatic-redux
It's entirely possible to write a React app with just props, local state and change handlers but once you have something of size the extra effort to integrate Redux greatly simplifies the chains of connections between your components making it easier to move things around as all the data flow is now decoupled from the exact hierarchy of your components
If nothing else I'd seriously recommend watching the Getting Started With Redux tutorial, it's a master piece of teaching. I quite like https://facebook.github.io/react/docs/thinking-in-react.html as well for getting you in the mood for the React way.
I made a game, it has penguins in it. It's pay what you like on Gumroad.
Currently Ebaying Nothing at all but I might do in the future.
I come from a Windows/WPF background, and Angular feels much more like that world to me than React does...but it's quite possible I just don't know React well enough yet.
PCA is actually not always great with very sparse data... I'm trying to remember what I used a couple years ago on something similar, I'll try to find it
Since I spoke out harshly about Angular, I should also clarify most of that is directed toward Angular 1. I don't have experience with Angular 2, and if I have aversion to it it might only be because the Angular brand left a bad taste in my mouth.
Yes, Angular 1 though, I feel is garbage, and I've never met an organization that did not express regrets about sticking to it. The best description I've heard of Angular 1 is "bandages over self inflicted wounds".
I don't know what Angular 2 looks like, but I wish anyone who chooses to work with it the best of luck, and I hope you're commitment to learning the framework pays off in the future for you and your loved ones.
I do most of my development these days in Angular 2, which is definitely a better framework.
In my view, Angular 1 is for sending data to a client-side web page and having it perform data-sensitive HTML rendering. End of list.
If you ONLY use it for this and you keep your architecture simple, it's a completely serviceable framework
When you start "developing in angular" that's right about where you fucked up
I'm probably putting words into the Angular 1 developers mouths, but after working in it for two years and managing a team of angular devs, I never got the impression they meant the framework to be the foundation for functionality-rich statically-served thin clients on the web. More like "a thing that lets you put ASP:Repeater workload on the browser instead of the server"
But now entire websites are built basically without any server at all, just a cloud API in a virtualbox somewhere and a JS client that consumes it.. Angular 1 is ill-equipped
Either way, all new web client dev is a trash fire now anyway. I'll be pumping out server rendered pages w/ CSS frameworks and web components till the day i die
god bless never having to have more than 100 concurrent users
I am definitely in the "cloud API in a container running in some EC2 instance somewhere" camp.
primes := [6]int{2, 3, 5, 7, 11, 13} var s []int = primes[1:4]We make an array of prime numbers, then a slice referencing that array. Slice starts at index 1, and then there's what I thought was either the length from the starting index, or the index of the end of the slice.
The slice is [3, 5, 7]. Does the syntax go "from index 1 to the fourth item"? Feels pretty counter-intuitive.
edit:
OK, so it goes up to but not including the number. Man that feels weird.
Sorry, I'm having a hard time writing a reply, since I just threw up all over my screen.
http://steamcommunity.com/id/pablocampy
I made a game, it has penguins in it. It's pay what you like on Gumroad.
Currently Ebaying Nothing at all but I might do in the future.
It is.
You've never used Javascript?
You may need to brush up on your Ruby slicing notations.
Not array slicing, at least. Never really had situations that called for it.
Well, it's regular syntax. It may feel unnatural, but it's pretty much the canonical to way to do it [star, end).
Yep! Still doing a lot of learning, want to get a firmer grasp of Go before I look at the codebase. Had a quick look and it felt like I had no idea what was native to Go, what was external libraries and what came from other parts of the codebase.
Good to hear! Hope it's a fun job.
Spend some time and get comfortable with your environment as well. Depending on what editor you are using, something like gocode is invaluable.
What do you mean?
You can also slice arrays by doing array[2,4] but that actually means something different: "starting at index 2 grab 4 items", and I don't think I've seen it used.
Right now I'm in Visual Studio Code and a go extension that seems fairly decent. Definitely open to alternatives.
edit: yeah, that extension uses gocode.
I may have misunderstood what you are saying. Golang syntax is a match for ..., I thought you are forgetting about it and thinking of .. only. My bad.
Use whatever works for you, but Vim & Emacs both excel for golang.
Join uuuusssssssssssssssss
http://steamcommunity.com/id/pablocampy
There, that's my vim knowledge
and then open it in nano
It's not as crazy as it sounds. A range in ruby is a representation of a range of values: 1..10 means "every number between one and ten, including ten", and 1...10 means "every number from one up to but not including ten". You really need both versions because 1...10 is not the same thing as 1..9 (9.5 is higher than 9 but less than 10, for example).
I get the functionality, but it seems like it would be so easy to make a mistake.
You also need to know to mash escape first to make sure it will accept the command!
Ctrl-z, kill %1 or killall vim is also acceptable