Porous tantalum scaffolds with the dodecahedron, G7, and cubic structures were fabricated by electron beam powder bed fusion (EB-PBF). The influence of pore structures on the deformation behavior and mechanical properties of porous tantalum scaffolds was investigated by compressive testing, tensile testing, three-point bending testing, and finite element analysis (FEA). In the compressive test, the deformation of G7 and cubic scaffolds was dominated by buckling. The dodecahedron scaffold was deformed mainly by bending, leading to its lower compressive strength. On the other hand, the dodecahedron scaffold had superior tensile and bending properties. The uniform maximum stress distributions of the dodecahedron scaffold in the tensile and bending tests contributed to the homogeneous structural deformation, which delayed the fracture of struts; whereas, the maximum stresses of G7 and cubic scaffolds were concentrated in the transverse struts, resulting in the fracture of transverse struts in the early deformation stage. Therefore, pore structures should be properly selected for EB-PBF manufactured porous tantalum scaffolds based on the requirements for mechanical properties.