can i use the jstl sql tag instead of this and is there any disadvantage in this tag.
Yes, you can, but I would strongly recommend you not to. JSP's role is not to shake hands with a database. You'd better go through a dedicated component (servlet, JSF backing bean...) to retrieve the data.
when does the connection close if i use this jstl sql tag
You should have already been discouraged from using these tags But you can still check the JSTL specification, 10.3 <sql:query> and 10.5 <sql:transaction>.