[ 3 / biz / cgl / ck / diy / fa / ic / jp / lit / sci / vr / vt ] [ index / top / reports ] [ become a patron ] [ status ]
2023-11: Warosu is now out of extended maintenance.

/biz/ - Business & Finance


View post   

File: 453 KB, 746x565, fb1.png [View same] [iqdb] [saucenao] [google]
18824671 No.18824671 [Reply] [Original]

>at home collecting big unemployment gibs check
>decide to learn some new skills because why not
>learn basic bitch web dev
>build a functional job board for niche jobs
>use free web scraping software to web crawl dozens of job boards for niche jobs related to my site. very easy to do, all data goes into excel sheet, spreed sheet can automatically upload into my job board how i want it arranged with little effort, after i curate the list

what the fuck am i missing here? seems like the better i get at web crawling i can build automated bots to gather all sorts of data, repackage it and sell it online in different ways. is it really this easy?

I'm not looking to make shit tons of money, only like 20-30k a year working on my own projects. seems very possible with how easy it is to do shit like this.

any advice?

>> No.18825531

>>18824671
Ask /g/

>> No.18825930

>>18825531
yeah, but i'm more interested to see how others have monetized web scraping. it's gotta be oversaturated, it's too easy to do. /g/ will just sperg out about some tech mumbo jumbo that i don't care about. anyone here ever successfully monetize web crawling? if so, how? seems like you have to repackage data in a very niche way aimed at a niche market and drive traffic that way

>> No.18825968

>>18824671
don't forget kneepads

>> No.18825978

>>18825930
You can web scrape or get APIs for crypto or stock prices then make money from Adsense

>> No.18826003

I automated the management of my fantasy football team because I got bored with logging into ESPN every week and swapping out Jaquarius Latomlinson III on his bye week.

>> No.18826047

>>18826003
Is that a real name

>> No.18826199

>>18826003
can you make this retard friendly to use on the surface and sell it as a service/app? they must exist already for fantasy football if possible. i don't know anything about fantasy football though, if it has a lot of third party applications, it must. espn doesn't let you automate the pickings? or whatever the fuck goes on with that game

>> No.18826232

>>18825531
do not go anywhere near /g/ if you want to be a successful developer.

OP it sounds like you have a brain. not a lot of us devs do.

>> No.18826305

>>18826047
I don't think so? Maybe though, who knows. Only like half of us in the league actually followed football lol.

>>18826199
Eh, I didn't want to do the extra work of monetizing it or actually reading ESPN's EULA, but maybe.

Fundamentally it was just some Selenium scripts in a Python wrapper, not sure how easily that can be converted into an app.

>> No.18826328

>>18824671
scraping is easy but you’ll never generate any search traffic with scaped content. maybe 5 years ago you could but google has cracked down on every SEO method imaginable so now only 10 year old authority sites with shitloads of high quality links show up in the serps.

>> No.18826489

>>18826328
yeah that's the conclusion i've been coming to overt time. it's been a really interesting ride so far though, learning about all this stuff. seems like the niche job sites that got in around 2007-2010 have an established foothold and make bank, even though some are fucking garbage.

check out flexjobsdotcom for example, absolute garbage site that just scraps data from indeed and reposts it. run by like 12 women, you gotta be kidding me. they charge suckers something like $14 a month for access to basic bitch shit i can already replicate all on my own, nearly fully automated, after a week or two of working on my site. these niggas are pulling in like 5+ million a year. what a fucking scam

>> No.18826538

>>18824671
How did you learn how to this stuff? What resources? Help a recently unemployed fag out

>> No.18826738

>>18826538
just go to youtube or udemy and type in "web development." kinda hard to know what is good though because there is so much content, suppose it depends on your style of learning. at a certain point (when you get basic concepts) i found it more useful to just start working on a project and wing it, even if i don't know much. just try to build it by searching shit on google when you get stuck. it's a constant fumble but over time you learn a lot more instead of just following along with some boring ass lecture.

you can also just go to webflow and make decent sites there without any coding. of course you are limited that way, but webflow is really good for basic shit, if you just want to use templates etc if you know the basics of html, css webflow is great. miles ahead of competitors like squarespace, weebly, wix

for the web crawling you can find all kinds of tutorials and even some decent templates like scrapy. also, there's octoparse, parsehub that require no coding and are user friendly, but i think you have to pay sub eventually.

i've been just piecing shit together, trying this mixed with that, trial and error. it's getting kinda fun, but i'm becoming increasingly doubtful i can easily monitize anything unless i get lucky or repackage stuff for some creative niche that not many people have thought of, hard to do. basically, you'd have to provide some kind of value or solve a problem, of course

>> No.18827357

>>18826232
nah i'm pretty much a retard who pieces shit together until it works. i'm too impatient to learn the details at a depth where i could be a paid developer. as a developer, do you think the whole 'learn to code' shit is a meme? seems like entry level jobs in "tech" like QA are overflowing with pajeets, what the fuck, is it all oversaturated now? i've been to a few tech campuses, facebook, paypal, google etc seems like its all chinks and pajeets walking around like they run the place, literally speaking mother fucking chinese to each other. QA seems easy like something you could teach kids straight out of highschool, who are in small depressed towns with no future, could give them a solid income to start off, but we have to import mass amounts of pajeets for this kind of thing? what would be the fastest/easiest skill to learn to get a foot in "tech" jobs?

>> No.18828300

>>18826738
>>18827357
Thanks anon

>> No.18829227

>>18827357
You're not wrong. If I had to estimate what proportion of H1B visas in use by my employer were actually necessary, I'd say less than 5% to actually scoop up the few geniuses that are truly what any country would want to hold onto. Everyone else is just...average. I don't have any individual hate for them, but they don't have any special talents that justify their visa either.

>> No.18830418

>>18829227
that's really depressing. imagine the demand for someone like me who's eager enough to learn some entry level shit, fuck i'd do QA if it meant i could get my foot in the door and learn some of these skills on the job. the demand would be so high that companies would pay people like me from east fuck small town to learn entry level stuff, each company would have their own bootcamp program with on the job training, paid. it's so twisted the amount of skills training and jobs that could be given to people who otherwise are doomed to wage cuck at starbucks or never get out of their opiate ridden town. i don't blame the foreigners either, i'd do the same thing in thier shoes. it's just a ploy for these big tech companies to get cheaper tech labor, it seems. i mean, perhaps i could get in some entry level position if i study my ass off and keep trying. maybe i'll try, we'll see. it just would be nice if the demand was as high as it should be for the locals

>> No.18830511

I’m still surprised that people get amazed by basic bitch html that I freaking learned on neopets, let alone other basics like css and js.
So yeah. It’s an odd thing, much like you fix a computer problem by googling something and your parents think your a computer genius for some reason even though it was basically nothing.
Add more to your arsenal if you can. Maybe find free classes and reverse engineer all ready made things to pick it apart to see what works and what doesn’t.

>> No.18830609

>>18827357
I’m not good at sitting and reading either.
I like treating it like a puzzle. It’s sort of why I took a class, I had the right answer a head of time though. HOWEVER I wanted to tweak it and go through everything. And see what worked and didn’t and why. Why did this original answer work but not what I put? Was there something wrong with the font? Did mess up the link?

>> No.18831018
File: 152 KB, 1365x2048, 92B60D96-A177-48BA-9C03-354B2E3102A8.jpg [View same] [iqdb] [saucenao] [google]
18831018

>>18824671
Bumping this thread because we need more entrepreneurial threads like this instead of gay pajeet memecoin shilling on this board.

I have nothing of value to add to this thread other than I’m more or less doing QA now by accident at my small firm.

>> No.18831168

>>18831018
>I have nothing of value to add to this thread other than I’m more or less doing QA now by accident at my small firm.
How did that happen anon

>> No.18831466

>>18830609
yes, exactly. it makes more sense to my fidgety brain if i can just do it hands on, try to make it work, even if it's not the "proper" way to do it or it's not how the guy in the video does it. then i'll go back and rewatch part of the lesson or read and have a much better understanding of how it works, or typically in my case, didn't work.

funny, this stay at home order has been great for me in some ways, i have a fellow bachelor housemate so i turned the garage into a little office adn i'll go back and fourth with my laptop between there and the living room couch, just working on stuff. I've never had the luxury of getting paid to stay home (unemployment for the win) i've been waging shitty jobs since i was 14. i don't have any defined schedule, i just work around the clock on my little projects at all hours day and night, sleep when i'm tired, eat when i'm hungry. i think i'm fueled by the dread of when this stay home with pay gig is up i'll have done nothing to take advantage of all the free time. I want to escape wage cucking very badly, so this free time to learn stuff is a blessing. i've gotten back into some day trading crypto after getting reck'd in 2018, i'm working out, grilling exotic meats. i know a lot of people got fucked by the rona chan but i wouldn't mind a few more months of this. saving 80% of my unemployment because i do jack shit all day besides work on projects. don't mean to rub it in but i think i'm ascending

>> No.18831556

>>18824671
If you want to graduate from using other tools to making your own, learn python, an html parsing lib like beautifulsoup and regex. For more complex automation and scraping javascript heavy pages you want to use selenium which gives you an api for controlling a web browser, extracting data from it and sending it commands (eg submitting forms, focusing a text field, etc)

>> No.18831990
File: 81 KB, 1000x800, E494DE5B-370F-4C12-A29F-C882B9DE783C.jpg [View same] [iqdb] [saucenao] [google]
18831990

>>18831168
I’m in IT Risk Management, which is just a fancy title for an IT auditor. I’m a consultant that helps companies get their shit together for HIPAA, Sarbanes-Oxley, and a myriad of other frameworks. On top of that practice, we have a little homegrown SaaS tool that helps you automate risk assessments for said frameworks. Basically all it does is dumb down the frameworks into easy to digest questions that normie middle management can answer.

The communication between the consulting firm and the software company is pretty nonexistent. I saw that my boss was going to do a webinar to shill the software to another client, so I asked if I could sit in just because I was curious about the tool and again, I didn’t really know a lot about it because bad communication, the boss loved my interest, that pays later.

Around a week later I’m working on a project that just so happens us using the tool to spit out a fancy report for one of our clients, I just so happen to accidentally stumble across a bug as I’m mindlessly answering questions. Boss sees the bug and asks if I want to help test. Fast forward to today and I’m plugged in with the dev team and have a dedicated machine now just for testing, all because I stumbled into a webinar, lol.

Moral of the story i suppose would be too always inquire and be hungry for new work.

>> No.18832010

>>18824671
I get headages following simple R tutorials. It feels exausting and utterly depressing having to look up every little shitty snipped of code.
Congratulations, you have the coder monkey gene. You are going to be somewhat rich