The answer to question number one has everything to do with people and politics and has little to do with refining costs. You are correct that diesel is simpler to refine and therefore slightly cheaper to produce. In theory, that would mean it would be cheaper for the consumers.
Now here is where the other factors come into play. Just like your example states, farm equipment and semi trucks all run on it. The people using this type of equipment are typically the producers of food, products, transportation of said items, etc.
Well politicians have found that they can jack up the fuel tax on diesel without really hurting demand, because people need the fuel to continue to produce/run their business/transport goods and therefore will keep using the fuel basically no matter the price. They need the fuel to continue making money.
With gasoline, the majority of the people using them are on the consumer side and therefore can more easily lessen their consumption of fuel when costs go up, ie carpool, group tasks involving driving, or stop going on unnecessary trips, eventually killing demand and actually potentially creating less total revenue from gasoline tax.
So people “in charge” of governments and their desires for constantly growing “tax revenues” to pay for various programs has far more to do with the cost of your fuel than the actual cost to produce, refine, and deliver it to you.
I’m not sure how to answer your second question. I’m guessing that the demand will lessen over time as a percentage of the total energy pie, but as populations grow and industry follows, it’s tough for me to say that the actual gallons of fuel needed to run this planet as it stands will ever actually go down.
I hope this made sense to someone other than myself.