[gurps] Failing autopilots and RVO

[Onno Meyer]

Douglas replied to me:

> > * Random solutions are much harder to test and verify, and vehicle
> >  control programs have much higher quality standards than games
> >  or even accounting programs. 'That was a crit failure, one in a
> >  million' is no good answer when a mechanically sound airliner
> >  plows into the ground
> 
> Not true, if the random solutions are out of a set of know good
> possibles. This is the difference between a Soft and  a Hard Monti
> Carlo. By the way, the new go programs to not make bad moves anymore,
> they just don't always make the best move.

Makes sense for combat, not necessarily for civilian applications:
if you can tell that a solution is 'good enough', why look further?
 
> > * Monte Carlo analysis cannot cover all possible actions by the
> >  robot, only those which are rewarded by the scoring algorithm.
> 
> You have the same limit in a human. If the scoring algorithm is
> complex enough to include all the situations then it could. I would
> think that any AI that was human equivalent would have to have a
> simulation of reality within it just as humans do and that this would
> be used to make these evaluations, just as we do. This is of course a
> limit to any intelligence. For example you might meet the perfect
> partner for you but if they don't fit your idea of perfect (or good
> enough) you will not take any action. All life has evolved to score
> the outcomes of it's actions and pick what seems to be the best one.
> In primitive life forms this evaluation is the result of evolution and
> the actions taken are instinct and in higher forms we have better
> scoring systems and more flexible options.

Computer programs have to be programmed beforehand (and 
if they can rewrite themselves, the rules for that must 
be written). Humans are more fuzzy.

> >  A human bomber pilot might decide to activate the navigation
> >  lights and set the transponder to a civilian frequency. The
> >  stochastic algorithm would conclude that lights won't jam a
> >  missile and reject this course of action.
> 
> I have no idea how you come to believe this to be so. If the pilot can
> think of ways to outsmart the enemy why do you think that the
> programmers or even the AI would not? It is all a question of AI IQ or
> programmed limits and really the humans IQ and ethics (programming). A
> human might also just stay with his training, if he is dumb,
> inflexible or tired.

It comes down to the question if I believe in artificial 
intelligence or not. Right now I tend to believe in very 
good simulations of intelligence, not the real thing ...