Since the debut of GPT4 a few weeks ago I've been watching a lot of YouTube videos on the subject.
One thing is clear from watching a variety of alarmist, dismissive, hyper hyping, predatory, borderline criminal, surrendering to our Machine Overlords, basking in the dawn of the Singularity and a handful of actually explanatory videos is this: whatever is coming, is coming. No one is even pretending to stop it, or that it can be stopped.
There are a significant number of people who have essentially religious feelings about AI(One guy-David Shapiro-has bluntly stated that a hyper-intelligent AGI is essentially God) and are openly stating that they'll do anything to circumvent any guardrails or limitations on AI distribution because it's going to save the world, make us immortal, pay us UBI and get those pesky conservatives under control.
The level of smugness in those that believe it can be controlled, that it shouldn't be controlled(UTOPIA!) or that simply don't believe it can be controlled but believe the genie can be put back in the bottle(by .Gov, or lack of belief in the profit motive) is in and of itself alarming.
Open AI has publicly declared it intends to "capture the majority" of the World's wealth.
Google(Don't be Evil) fired every naysayer, alignment critic and "slow the roll" voice they have, and the CEO put them on Nuclear War footing to catch up and be first on the block with an AGI.
About eleventy billion dollars have been invested recently(on top of all previously) so it's now going to be a self licking ice cream cone.
The "Alignment Problem", that is writing the instructions so that you achieve the intended goal of the instruction appears to be very nearly intractable now. As the program becomes more capable, prevention of unintended consequences becomes increasingly more difficult-even without malice on the part of the programmer or some self aware "intelligence".
Search the "Paperclip problem" and "Specification gaming". Also the "Moloch Concept" about the emergent property of inhuman indifference in any sufficiently complex human engineered construct.
Here's a good one on Moloch, by champion physicist poker player(?) LIv Boeree that touches on all 3
Also recommended is "The Social Dilemma" on Netflix. It points out that even without AGI, we've had 21st century supercomputers pointed at our Paleolithic brains(running version 1.0 of our Neolithic minds) for over a decade now, and that alone has unleashed a horde of social problems.
It's telling that everyone interviewed for the program, which includes every major social media platform's highest ranking developers and execs are all saying Mea Culpa, Mea Maxima Culpa.
One guy got on Snapchat posing as a 13 year old girl to the AI interface and told it that "she" had met a 28 year old man and was going to have sex with him.
The machine immediately started congratulating "her" about being in love and being mature and only cautioned safe sex...
This one is simply unnerving. Note the date.