Why is SVM so powerful?

Can you elaborate on why SVM, or Support Vector Machine, is considered such a powerful tool in the realm of machine learning? What are the key factors that contribute to its effectiveness, and how does it compare to other popular algorithms in terms of performance and efficiency? I'm particularly interested in understanding the mathematical underpinnings that make SVM so adept at handling complex classification and regression tasks. Additionally, are there any limitations or scenarios where SVM might not be the ideal choice?