The short answer is "Absolutely not!"
First, firms generally understand that candidates are interviewing at multiple firms during the lateral job-search process. Thus, firms understand that a candidate may have to see the interview process through to completion before giving an answer on an offer.
Now, having said this, it is equally important to keep in mind that the firm with whom you are interviewing is likely considering other candidates as well. In order to be fair to all parties involved, while you do not have to give an answer on the spot, you should not let an offer sit for so long that you prejudicing the firm's ability to consider other candidates. Absent extraordinary circumstances, you should respond to the firm's offer within two weeks of the date you receive the offer.