The new forums will be named Coin Return (based on the most recent vote)! You can check on the status and timeline of the transition to the new forums here.
The Guiding Principles and New Rules document is now in effect.
Help me become a better Software Developer/Engineer
Jimmy King
Registered User regular
Registered User regular
tldr at bottom for those who don't want to read.
So first some background would be helpful, I guess. As you may have picked up on threads I've posted in about IT careers or programming, I'm a self taught developer. I started programming on and off using qbasic and then C in the mid-90s before college. I went to a state university for a year in their CS program, found the first few programming classes very easy and got A's without attending class for longer than was required to turn in the assignment and get the next one, etc. in the late 90s. I dropped out after the first year anyway because I hate college and was overall a pretty terrible student with no discipline.
As time went on I worked my way through IT grunt work into programming jobs. I also returned to school at a community college where I don't actually learn anything in the programming or other computer classes because they only covers concepts I already understood 10-15 years ago. I've also worked my way up to being Director of IT for a small international corp. where I am the primary developer (with one jr level guy I work with) for our core application, a large content management system for content for mobile phones, delivery of content to them, etc. I design the new features and fixes and I implement them. I also design new applications for the company, primarily web based apps for various functions, but sometimes other stuff.
Overall I feel I'm pretty good. Average at least, but certainly not super amazing. I primarily write Perl with occasional C# here at work. I use Perl and Java mostly for personal projects with some C thrown in for good measure on occasion just to stay on top of things (I can handle pointers, work with linked lists, etc). I mostly write web apps but also do some console apps, the occasional simple game (both text or console/ncurses based and 2d graphics), and some Android development. There's a good bit of SQL, too, with the web apps and Android (it uses SQLite for most persistent data storage).
Having missed out on the formal education, though, I can't help but wonder what I didn't learn. What am I missing that could take me from being a good developer/engineer to an amazing one? Some stuff I know I should know more about, but just haven't learned more about yet such. I've never used and couldn't name an OO design pattern other than the occasional singleton and simple factories. I've never used a right join or a union in SQL and couldn't tell you why you would use a right join over a left join or think of any sane DB design that would require the use of a union. I know nothing of sorting algorithms and couldn't write you anything other than a bubble sort or a quicksort that I can't guarantee would actually be efficient. What else is there that I don't even know I don't know?
I've got a solid grasp on programming basics, standard OO stuff, and related stuff such as networking, network protocols, etc. which I feel is important to know given the direction apps are going with being web based or otherwise using a network. Like I said earlier, I feel I'm at least average ability at designing and implementing moderately complex systems based on my experience here and personal projects and time I have spent with other developers. I'm just not super amazing. I'm not going to write my own simple OS or my own programming language or compiler for an existing language or any cool stuff like that with my current knowledge and skills.
tldr; Experienced self taught programmer wants to learn programming and software design/engineering things that may have been missed due to lack of formal education.
Recommend me things that I should learn about. Recommend me good books to read and/or web sites to learn about this stuff.
So first some background would be helpful, I guess. As you may have picked up on threads I've posted in about IT careers or programming, I'm a self taught developer. I started programming on and off using qbasic and then C in the mid-90s before college. I went to a state university for a year in their CS program, found the first few programming classes very easy and got A's without attending class for longer than was required to turn in the assignment and get the next one, etc. in the late 90s. I dropped out after the first year anyway because I hate college and was overall a pretty terrible student with no discipline.
As time went on I worked my way through IT grunt work into programming jobs. I also returned to school at a community college where I don't actually learn anything in the programming or other computer classes because they only covers concepts I already understood 10-15 years ago. I've also worked my way up to being Director of IT for a small international corp. where I am the primary developer (with one jr level guy I work with) for our core application, a large content management system for content for mobile phones, delivery of content to them, etc. I design the new features and fixes and I implement them. I also design new applications for the company, primarily web based apps for various functions, but sometimes other stuff.
Overall I feel I'm pretty good. Average at least, but certainly not super amazing. I primarily write Perl with occasional C# here at work. I use Perl and Java mostly for personal projects with some C thrown in for good measure on occasion just to stay on top of things (I can handle pointers, work with linked lists, etc). I mostly write web apps but also do some console apps, the occasional simple game (both text or console/ncurses based and 2d graphics), and some Android development. There's a good bit of SQL, too, with the web apps and Android (it uses SQLite for most persistent data storage).
Having missed out on the formal education, though, I can't help but wonder what I didn't learn. What am I missing that could take me from being a good developer/engineer to an amazing one? Some stuff I know I should know more about, but just haven't learned more about yet such. I've never used and couldn't name an OO design pattern other than the occasional singleton and simple factories. I've never used a right join or a union in SQL and couldn't tell you why you would use a right join over a left join or think of any sane DB design that would require the use of a union. I know nothing of sorting algorithms and couldn't write you anything other than a bubble sort or a quicksort that I can't guarantee would actually be efficient. What else is there that I don't even know I don't know?
I've got a solid grasp on programming basics, standard OO stuff, and related stuff such as networking, network protocols, etc. which I feel is important to know given the direction apps are going with being web based or otherwise using a network. Like I said earlier, I feel I'm at least average ability at designing and implementing moderately complex systems based on my experience here and personal projects and time I have spent with other developers. I'm just not super amazing. I'm not going to write my own simple OS or my own programming language or compiler for an existing language or any cool stuff like that with my current knowledge and skills.
tldr; Experienced self taught programmer wants to learn programming and software design/engineering things that may have been missed due to lack of formal education.
Recommend me things that I should learn about. Recommend me good books to read and/or web sites to learn about this stuff.
Jimmy King on
0
Posts
If you're happy with your current level of skill and believe that you will be happy to do a similar type of job as what you're doing now for the foreseeable future, then there's little point in learning more. More importantly, you won't be motivated to learn more.
After all, if you're essentially a director of IT based on your current IT skills, perhaps IT skills aren't what you should be pursuing. Maybe what would get you further are other skills, such as management. For example, management isn't about just getting other people to do your work -- you've got a dude under you right now, so you know that's not the case. Maybe you'd get a greater handle on project management, on pitching ideas, on where areas can be improved, where there's no cost benefit to improving the efficiency of a program.
In other words, maybe what you need to learn isn't just more coding, but some of the business elements behind coding. I say that because you said you worked up to director of IT, which doesn't strike me as something someone would do who just wanted to write code all day working on individual deliverables.
That said, it might be useful if you aren't really sure of the specifics to look up algorithm analysis and big O notation, which is used to describe how "fast" a particular algorithm is with respect to something or other (linear time in the length of the list, etc). Also, learning a lot about data structures will help you make the right choices when doing your programming, although with that much experience it should be pretty obvious when you need to use a dictionary/map over a list/array. In particular, it will help you when you are reading the MSDN documentation on the difference between a SortedList and a SortedDictionary. When it says that in general, insertion to a sorted list is O(n) and for a SortedDictionary it is O(log n), you will know that the sorted list is probably implemented using an array similar to how the List is implemented, whereas the sorted dictionary is probably implemented using a binary tree.
After that, check out The Mythical Man Month. It's a very interesting read and goes into why throwing developers on a project to try and finish it faster is a terrible idea, along with other things.
Both of these are great for people who manage software developers as well, the latter in particular.
EDIT - One other note: I would highly recommend getting more in-depth understandings of OO and what it really means in the software world. It's interesting on paper, but seeing how it applies to enterprise-level apps is where you'll really start to see the benefits. .NET is built around this design paradigm, as are other popular languages like Java and Ruby.
Also, join a coding help message board and read some topics, impart knowledge and see what alternate solutions are proposed.
I want to be able to write my own simple OS, or contribute to Linux kernel development, or Apache, or write my on IP stack, etc. At the same time I want to be better at designing large scale apps. that are scalable, more easily added to and modular, etc.
I'm not terribly interested in the management side of things at all and I just happened to kind of stumble into the current role and mainly accepted it because my actual job I do wouldn't change much, but it sounds good on a resume and gives me a little more pull with the "title is everything" company politics sort of people that I have to deal with from time to time. I do programming because it's something I enjoy and want to be better at it just because I can be and I find it interesting. I don't just do it for work, I do it as a hobby in my spare time at home the same way some people do jigsaw puzzles or whatever. Programs are my jigsaw puzzles in many ways, really.
Mythical Man Month I defiinitely need to read, too. I see it brought up a lot and understand (and agree with based on what I've seen in my own experience) the basic premise, but it would be interesting and useful to really see the details and arguments the author offers up.
I think I've got a pretty solid understanding of OO including enterprise stuff, not that I know everything as that's the whole point of this thread. The core application that I do development work on is a pretty large CMS system which has a database with 130 tables at last count, tables with 10s of millions of rows, cgi interfaces that make tcp connections to a back end application server, over 120 in house developed Perl modules plus cpan modules and the actual scripts/applications that use them, and handles a number of requests per day at least well into 6 digits... not sure of exact numbers, but lots. The DB is handling around 600 requests/second (but one request on the web interface can result in many db requests). So not large bank network and computer systems type stuff, but maybe a good bit larger than people might think when they read "small company". That's also just the core app, there are also other smaller but still decent sized apps for reporting, a consumer facing modular CMS that allows you just write and drop in modular plug ins I've designed from the ground up, etc.
I didn't design the core system, just inherited it, and even with my current knowledge there are a lot of things I would do differently if given the opportunity to rewrite that system from the beginning.
I'm sure I do still have a good bit to learn here in terms of designing systems well, though. I don't really know what to look for or where to look for it, though. Pretty much everything I read about OO goes over the same basic principles that I understand well.
How many user groups do you attend? How active are you in your community? I'm not big time enough to be invited to speak at local events yet, but internally to my company, I present fairly regularly on interesting things I'm learning on my own time.
Where do you see yourself in 10 years? Just like when starting to write a large application, the end seems insurmountable - there's just so much to do, and so much to comprehend at first. Getting your architecture implemented, then creating your business logic on top - it's a lot of work. But knowing where you are going gives you more direction.
As for the college experience... it leaves some stuff to be desired. I couldn't tell you how to write a merge sort, but I know what it does. I'll just look it up on the internet. I couldn't write a good semaphore in college, but I know the basics of multi threading, and how to implement it in .NET. Sometimes a level of abstraction can be a good thing, because otherwise you are too concerned in the minutiae. At the end of the day, you are benefiting from decades of abstraction so that you aren't coding in assembly.
I do have a formal CS education and I've found a number of things that I learned to be helpful. For example, low level stuff like:
- Learning how a computer works from the transistor level up to 4th generation programming languages,
- Learning what a flip-flop is, how to create some basic logic circuits and so on. There is educational software out there that you can use to build this stuff in a software simulation.
- Learning assembler. We did it on a simulated Motorola 68K processor which was great, because it's a very limited command set.
- Learning to write software in pure C to learn memory management and developing your own data structures (linked lists, trees, etc.).
- Learning about memory hierarchies and caching.
- Learning about OSes. Modern Operating Systems by Tanenbaum is the essential book. We had programming assignments to develop parts of an OS like a shell, memory manager, file system. We used the NachOS project as an environment, but to be honest, I never turned any of these assignments in because they were too complex for me to invest the time.
- Learning relational algebra and the finer points of SQL although, as you point out, the more complex stuff is irrelevant in practice.
Higher level stuff:
- It can make sense to try and work with older, limited programming languages that emphasize certain concepts. So try stuff like pure C, Haskell, Gofer, Prolog, Lisp. It was painful and I wouldn't want to work on a multi-million lines of code project in any of them, but I learned a lot about different programming paradigms and how they apply to different problems
- Learning about recursion, when to use it etc.
- Algorithm theory: What does it mean for an algorithm to be correct, when is it complete? How do probabilistic algorithms work?
- OO concepts / design patterns. Understanding these greatly helped me in understanding how reusable software is built, but I only really understood them when I started working on a bigger project and saw them used.
Note than of everything that I listed only the last item (OO concepts) is directly applicable to my current work. Overall, if you want to become an amazing developer, I think you need to work in a professional team on a suitable project. I have been working on large scale enterprise software for the last four years and was lucky in starting in a project that was completely new development. So I had the chance to help design a large framework from the ground up with some very talented and experienced people. This has done wonders for my architecture skills.
When I was looking for a job, I also had an offer for embedded software for automotive systems, which would probably have been a great fit if I had wanted to work on more low level, algorithmic, hardware-dependent stuff.
My standard development style is relatively close to test driven design, though, at least based on the high level overview I read.
I don't currently do any of this. For work, our local office is small. It's me and a couple of business guys, so no one to really present to. I've got one jr level developer (I really should quit referring to him that way. He's more of a sys admin sort and does jr level dev stuff, too) that's on the other side of an ocean.
As to the community, perhaps I should look into the local dev community and see what's up. I have no idea if there's anything or not. My assumption is not much since I am friends with several developers on both the Linux and Windows side of things and they have never mentioned such things. We do have a LUG or two that might be good to get involved with, though.
Another very good question and one I'm not completely sure of myself. I like the small to mid sized company thing like I've got going now where I get to make decisions on both the overall system design and the low level details. I find both very interesting. Based on my current experience I'd really like to be able to be able to write higher performance code and lower level stuff - rewrite parts of Perl modules in C, write Apache modules, etc. and be a little better at planning ahead and ensuring that design decisions don't back me into corners that require to later have to write code that I consider bad because it's the only way to do whatever needs done without rewriting tons of previous stuff. I get better at the latter as time goes on and can see it in my own work, but would still love to do some more formal reading on it rather than relying purely on learning from my mistakes or other people's mistakes that I've inherited in a system.
Due to my current experience I'm headed in a more mobile phone oriented direction, but that can still be web back end stuff like I do now (managing content for mobile phones, delivering to them, generation of mobile pages) and mobile applications.
This may be the case as long as you are still working where you are now, but I assure you, if you have to leave, you will learn very quickly that all the experience in the world doesn't trump a degree in this job market because a degree is very easy to quantify and search for in a stack of resumes.
The book, along with the other books mentioned previously, are on my amazon wishlist so that I can pick them up at some point.
Good stuff here and I'll definitely be hitting the books... or google, really, when I get home this evening.
And use it exclusively.
I'll wager that within a couple of weeks you'll have caught the bug. Linux is probably the best do-it-yourself teaching/learning tool for low level things. As you play with it more and more, you get deeper and deeper.
Get something easy to start out with, like Ubunut, with a GUI and all. I can't quantify how long it might take you, but eventually you'll let your computer sit there for days as you're compiling your own OS based on it. Microsoft might be the standard, but one of the reasons why many universities use Linux is because you can get really, really low level. You can make programs and play around with the actual registers on the processor.
If you want to learn how all of those work on a hardware/concept level, like the flip-flops mentioned, or OR gates and the like, there are many books on things like that.
There's a book I used when I took my Linux OS class at the U that started out with basic concepts of how computers worked, how the Van-Neumann architecture that's in PC's nowadays came about, the whole history, lots of fluff like that, and eventually worked it's way up to having you program your own OS.
Definitely more what I'm looking for here. I understand how computers work, cpu registers, etc. but for instance have never heard of this Van-Neumann architecture. Any chance you remember the name of the book you used? I've read plenty of technical books due to my return to school and for the most part they are total crap and filled with retarded garbage such as (this is nearly word for word), "Scripting language like PHP and JavaScript have very little processing power and instead rely on passing data into system binaries to get their output". These books all have good reviews on Amazon, so I don't exactly trust the online reviews to pick out good books. From here, though, I can see who knows what they're talking about and if they recommend something, I can be pretty sure it's good. If it's not, someone is probably going to point out that it's not such a good book.
I probably still have the book. I don't have it in front of me now, so I can't recall offhand what the name of it is. It's just one of those standard text books for class. I can give you more information tonight, like the title, edition, author, all of that.
np, the lack of Linux experience is often a pretty safe assumption. I don't think I mentioned anything platform specific other than .Net either, which although can be done on Linux, is safe to assume a person means on Windows unless they say otherwise.
The info on the book would be awesome when you have it available. I'm a very hands on learner and get a lot out of being able to take a working piece of code and tearing it apart to see what it does and why it does it that way, so a book that walks me through the beginnings of writing a useable OS seems like it would do a lot for me.
Known as one of the better schools for CS in the country, but leaning heavily towards theory. Fun fact: You don't need to write a single line of code to graduate. They always told us to please go and learn how to program on our own time because it's so important. But they didn't want coding to take away any course time from their precious theory.
But I enjoy having a broad theoretic base. I always get all the attention at parties when I say that I once knew how to prove Gödel's Incompleteness Theorem.
not really
Jimmy King, I actually have two books. A cursory glance doesn't reveal much, but my parsing through it hasn't revealed a whole ton, and I'm sure you can find a much more concise summary on the internet once I give you the names.
I have Computer Systems: A Programmer's Perspective by Randal E. Bryant and David R. O'Hallaron. I believe this one is the one that ultimately builds the OS, judging from the chapter titles and several of the pages, though I can't find anything specific without starting to read the whole book. This one has you build a complete linker, utilize the cache directly, and goes deeply into the computer architecture.
The other one I have is UNIX Systems Programming: Communication, Concurrency and Threads by Kay A. Robbins and Steven Robbins. This one seems to be mostly "how to program POSIX things." The title is very descriptive.
Remember, these are books I used in college, so I don't think you'll have a problem finding an old edition. It's not like the x86 computer architecture has changed in the last XX years....
And I'd definitely read up on algorithms. http://code.google.com/p/graph-theory-algorithms-book/
See how many books I've read so far in 2010
I need to check on The Pragamatic Programmer, but I think I've got that one on my list already. I'll get to reading that algorithm book online asap.
As to returning to school, I have actually gone back and am very, very slowly working towards a degree starting with community college. 1-2 classes/semester takes a long time, though, and community college programming classes aren't exactly challenging or presenting anything I haven't been well versed in for 10+ years. Some day I'll be taking classes from a 4 year school, but that's a bit off right now, so I'm looking for books and the like so I can do it at my own pace, in whatever order I want, etc.